Electronic devices, programs, and systems

Abstract

This electronic device comprises: a communication unit that communicates with a speaker's terminal; a speech output unit that outputs a speaker's speech signal received from the speaker's terminal by the communication unit as a speaker's speech; and a control unit that sets the volume of the speech outputted from the speech output unit. The control unit transmits information visually indicating the level of the speech at the position of a candidate to which the speech is to be transmitted, and can change the volume of the speech when the level of the speech is changed at the terminal.

Description

Cross-reference to Related Applications

[0001] This application claims the priority of Japanese Patent Application No. 2021-115943, filed in Japan on July 13, 2021, and incorporates the entire disclosure of the prior application herein by reference.

Technical Field

[0002] This disclosure relates to electronic devices, programs, and systems.

Background Art

[0003] In recent years, opportunities to conduct so-called remote meetings, such as Web conferences or video conferences, have been increasing. In a remote meeting, an electronic device (or a system including an electronic device) that realizes communication among participants present at multiple locations is used. For example, when a meeting is held in an office, assume a scenario where at least one of the meeting participants conducts a remote meeting from home at a remote location. In this case, the audio and / or video of the meeting in the office is acquired by, for example, an electronic device installed in the office and transmitted to, for example, an electronic device installed at the participant's home. Also, the audio and / or video at the participant's home is acquired by, for example, an electronic device installed at the participant's home and transmitted to, for example, an electronic device installed in the office. According to such an electronic device, a meeting can be held without all participants gathering at the same place.

[0004] Various technologies applicable to the above-described remote meetings have been proposed. For example, Patent Document 1 discloses a device that overlays and displays a figure representing the output range of a directional sound output by a speaker on an image captured by a camera. According to this device, the output range of the directional sound can be visually grasped.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

[0006] An electronic device according to one embodiment is A communication unit that communicates with the speaker's terminal, The communication unit includes an audio output unit that outputs the speaker's voice signal received from the terminal as the speaker's voice, A control unit that sets the volume of the sound output by the sound output unit, It is equipped with. The control unit transmits information to the terminal that visually indicates the level of the sound at the location of a candidate target to which the sound is transmitted, and when the level of the sound changes in the terminal, the volume of the sound can be changed.

[0007] A program according to one embodiment is: In electronic devices, The steps involve communicating with the speaker's device, The steps include outputting the speaker's voice signal received from the terminal as the speaker's voice, The steps include setting the volume of the sound to be output in the output step, The steps include transmitting to the terminal information that visually indicates the level of the sound at the location of a candidate target to which the sound is to be transmitted, When the audio level is changed in the terminal, the volume of the audio can be changed. Make it run.

[0008] The system according to one embodiment is This includes electronic devices and speaker terminals that are capable of communicating with each other. The aforementioned electronic device is A voice output unit that outputs the speaker's voice signal received from the terminal as the speaker's voice, A control unit that sets the volume of the sound output by the sound output unit, It is equipped with. The control unit transmits information to the terminal that visually indicates the level of the sound at the candidate location of the target to which the sound is transmitted, and controls the terminal to change the volume of the sound when the level of the sound is changed. The aforementioned terminal is A sound collection unit that collects the voice of the aforementioned speaker, A control unit transmits the speaker's voice signal to the electronic device, receives information from the electronic device that visually indicates the level of the voice at the location of a candidate target to which the voice is to be transmitted, and controls the electronic device to transmit an input to change the level of the voice. It is equipped with.

[0009] Furthermore, the electronic device according to one embodiment is The device is configured to communicate with the speaker's terminal and other electronic devices that output the speaker's voice signal as the speaker's voice. The electronic device includes a control unit that sets the volume of the sound that the other electronic device amplifies. The control unit transmits information to the terminal that visually indicates the level of the sound at the candidate location of the target to which the sound is transmitted. The terminal can change the volume of the sound when the level of the sound is changed, and controls the other electronic devices to output the sound.

[0010] Furthermore, a program according to one embodiment is: In electronic devices, The steps include communicating with the speaker's terminal and other electronic devices that output the speaker's voice signal as the speaker's voice, The steps include setting the volume of the sound output by the other electronic device, The steps include transmitting to the terminal information that visually indicates the level of the sound at the location of a candidate target to which the sound is to be transmitted, The steps include: changing the volume of the audio when the audio level is changed in the terminal; The steps include controlling the other electronic device to output the sound, Make it run.

[0011] Furthermore, the system according to one embodiment includes the speaker's terminal, an electronic device, and other electronic devices. The terminal and the electronic device are configured to be communicable with the other electronic devices. The terminal includes a sound collection unit that collects the speaker's voice, and a control unit that transmits the speaker's voice signal to the other electronic devices, receives information visually indicating the level of the voice at the position of the candidate for the target to which the voice is transmitted from the other electronic devices, and controls to transmit an input to change the level of the voice to the other electronic devices. and is provided with. The electronic device includes a voice output unit that outputs the speaker's voice signal received from the other electronic devices as the speaker's voice, and is provided with. The other electronic device includes a control unit that sets the volume of the voice output by the electronic device, and the control unit transmits information visually indicating the level of the voice at the position of the candidate for the target to which the voice is transmitted to the terminal, can change the volume of the voice when the level of the voice is changed in the terminal, and controls the electronic device to output the voice.

Brief Description of Drawings

[0012] [Figure 1] It is a diagram showing an example of a usage mode of a system including an electronic device and a terminal according to one embodiment. [Figure 2] It is a functional block diagram schematically showing the configuration of an electronic device according to one embodiment. [Figure 3] It is a functional block diagram schematically showing the configuration of a terminal according to one embodiment. <​​​​​​ [Figure 6] This figure shows an example of a display by a terminal according to one embodiment. [Figure 7] This is a flowchart illustrating the operation of an electronic device according to one embodiment. [Figure 8] This figure shows an example of a display by a terminal according to one embodiment. [Figure 9] This figure shows an example of a display by a terminal according to one embodiment. [Figure 10] This is a flowchart illustrating the operation of an electronic device according to one embodiment. [Figure 11] This figure shows an example of a display by a terminal according to one embodiment. [Figure 12] This is a functional block diagram schematically showing the configuration of an electronic device according to another embodiment. [Figure 13] This is a functional block diagram schematically showing the configuration of an electronic device according to another embodiment. [Figure 14] This diagram shows an example of how a system including electronic devices, terminals, and servers according to one embodiment is used. [Figure 15] Figure 14 is a functional block diagram that schematically shows the configuration of the server and other components. [Figure 16] Figure 14 is a flowchart illustrating the operation of servers and other components. [Modes for carrying out the invention]

[0013] In this disclosure, “electronic device” may mean a device powered by electricity supplied from, for example, a power grid or a battery. In this disclosure, “user” may mean a person who uses or can use an electronic device according to one embodiment (typically a human), and a person who uses or can use a system including an electronic device according to one embodiment. In this disclosure, a meeting in which at least one participant participates by communication from a different location from the other participants, such as a web conference or video conference, is collectively referred to as a “remote conference.”

[0014] There is a growing need for further improvements in the functionality of electronic devices that enable communication between multiple locations, such as in remote meetings. The purpose of this disclosure is to improve the functionality of electronic devices, programs, and systems that enable communication between multiple locations. According to one embodiment, the functionality of electronic devices, programs, and systems that enable communication between multiple locations can be improved. Hereinafter, an electronic device according to one embodiment will be described in detail with reference to the drawings.

[0015] Figure 1 shows an example of how an electronic device is used according to one embodiment. Hereinafter, we will explain the scenario assuming that participant Mg participates remotely from home RL in a meeting held in conference room MR, as shown in Figure 1. As shown in Figure 1, participants Ma, Mb, Mc, Md, Me, and Mf participate in the meeting in conference room MR.

[0016] As shown in Figure 1, the electronic device 1 according to one embodiment may be installed in the conference room MR. On the other hand, the terminal 100 that communicates with the electronic device 1 according to one embodiment may be installed in the participant Mg's home RL. The location of participant Mg's home RL may be different from the location of the conference room MR. The location of participant Mg's home RL may be far away from the location of the conference room MR, or it may be close to the location of the conference room MR.

[0017] As shown in Figure 1, the electronic device 1 according to one embodiment is connected to the terminal 100 according to one embodiment, for example, via a network N. The electronic device 1 according to one embodiment may be connected to the terminal 100 according to one embodiment by at least one of wireless and wired means. In Figure 1, the connection between the electronic device 1 and the terminal 100 via the network N by wireless and / or wired means is shown by dashed lines. In one embodiment, the electronic device 1 and the terminal 100 may be included in the remote conferencing system according to one embodiment.

[0018] In this disclosure, the network N shown in Figure 1 may appropriately include, for example, various electronic devices and / or devices such as servers. The network N shown in Figure 1 may also appropriately include, for example, devices such as base stations and / or repeaters. Furthermore, in this disclosure, when, for example, electronic device 1 and terminal 100 "communicate," it may be assumed that electronic device 1 and terminal 100 communicate directly. Furthermore, when, for example, electronic device 1 and terminal 100 "communicate," it may be assumed that electronic device 1 and terminal 100 communicate via at least one of other devices and / or base stations. Furthermore, when, for example, electronic device 1 and terminal 100 "communicate," more specifically, it may be assumed that the communication unit of electronic device 1 and the communication unit of terminal 100 communicate.

[0019] The above notation may include the same intent not only when electronic device 1 and terminal 100 "communicate", but also when one "transmits" information to the other, and / or when the other "receives" information transmitted by the other. Furthermore, the above notation may include the same intent not only when electronic device 1 and terminal 100 "communicate", but also when any electronic device communicates with any other electronic device.

[0020] In one embodiment, the electronic device 1 may be arranged in a conference room MR as shown in Figure 1, for example. In this case, the electronic device 1 may be positioned to acquire the audio and / or video of at least one of the conference participants Ma, Mb, Mc, Md, Me, and Mf. Furthermore, the electronic device 1 outputs the audio and / or video of participant Mg, as described later. For this reason, the electronic device 1 may be positioned so that the audio and / or video of participant Mg output from the electronic device 1 reaches at least one of the conference participants Ma, Mb, Mc, Md, Me, and Mf.

[0021] In one embodiment, terminal 100 may be positioned in the participant Mg's home RL in a manner such as shown in Figure 1. In this case, terminal 100 may be positioned in a location where it can acquire the participant Mg's voice and / or video. Terminal 100 may acquire the participant Mg's voice and / or video using a microphone or headset and / or camera connected to terminal 100.

[0022] Furthermore, as described later, terminal 100 outputs the audio and / or video of at least one of the meeting participants Ma, Mb, Mc, Md, Me, and Mf in the conference room MR. For this reason, terminal 100 may be positioned so that the audio and / or video output from terminal 100 reaches participant Mg. The audio output from terminal 100 may be positioned so that it reaches participant Mg's ears via, for example, headphones, earphones, or a headset.

[0023] Figure 1 shows just one example of how the electronic device 1 and terminal 100 according to one embodiment may be used. The electronic device 1 and terminal 100 according to one embodiment may be used in various other ways.

[0024] The remote conferencing system, including the electronic device 1 and terminal 100 shown in Figure 1, allows participant Mg to behave as if they were participating in a meeting held in a conference room (MR) while remaining at home (RL). Furthermore, the remote conferencing system, including the electronic device 1 and terminal 100 shown in Figure 1, allows meeting participants Ma, Mb, Mc, Md, Me, and Mf to feel as if participant Mg is actually participating in the meeting held in the conference room (MR). In other words, in the remote conferencing system including the electronic device 1 and terminal 100, the electronic device 1 placed in the conference room (MR) can act as an avatar for participant Mg. In this case, the electronic device 1 may function as a physical avatar representing participant Mg. Alternatively, the electronic device 1 may function as a virtual avatar displaying an image of participant Mg or an image of participant Mg, for example, a characterized version of participant Mg.

[0025] Next, the functional configuration of the electronic device 1 and terminal 100 according to one embodiment will be described.

[0026] Figure 2 is a block diagram schematically showing the functional configuration of the electronic device 1 shown in Figure 1. Below, an example of the configuration of the electronic device 1 according to one embodiment will be described.

[0027] The electronic device 1 according to one embodiment can be various types of devices, but for example, it may be a device specifically designed for this purpose. Furthermore, the electronic device 1 according to one embodiment may include, for example, a general-purpose smartphone, tablet, phablet, notebook computer (notebook PC or laptop), or computer (desktop) connected to a device specifically designed for this purpose.

[0028] As shown in Figure 2, the electronic device 1 according to one embodiment may include a control unit 10, a storage unit 20, a communication unit 30, an imaging unit 40, a sound collection unit 50, an amplifier 60, an audio output unit 70, a direction adjustment unit 80, and a display unit 90. In one embodiment, the electronic device 1 may not include at least some of the functional units shown in Figure 2, and may include components other than the functional units shown in Figure 2.

[0029] The control unit 10 controls and / or manages the entire electronic device 1, including each functional unit that constitutes the electronic device 1. The control unit 10 may include at least one processor, such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor), to provide control and processing capabilities for performing various functions. The control unit 10 may be implemented as a single processor, as several processors, or as separate processors. The processor may be implemented as a single integrated circuit (IC). The processor may be implemented as a plurality of communicably connected integrated circuits and discrete circuits. The processor may be implemented based on various other known technologies.

[0030] The control unit 10 may include one or more processors and memory. The processors may include general-purpose processors that load specific programs and execute specific functions, and dedicated processors specialized for specific processing. The dedicated processors may include application-specific integrated circuits (ASICs). The processors may include programmable logic devices (PLDs). The PLDs may include field-programmable gate arrays (FPGAs). The control unit 10 may be either a system-on-a-chip (SoC) or a system-in-a-package (SiP) in which one or more processors cooperate. The control unit 10 controls the operation of each component of the electronic device 1.

[0031] The control unit 10 may be configured to include, for example, at least one of software and hardware resources. In addition, in the electronic device 1 according to one embodiment, the control unit 10 may be configured by specific means in which software and hardware resources cooperate. The amplifier 60 described below may also be configured to include at least one of software and hardware resources. In addition, in the electronic device 1 according to one embodiment, at least one of the other functional units may also be configured by specific means in which software and hardware resources cooperate.

[0032] In one embodiment of the electronic device 1, the operations such as control performed by the control unit 10 will be described further later.

[0033] The storage unit 20 may function as a memory for storing various types of information. For example, the storage unit 20 may store programs executed in the control unit 10, and the results of processes executed in the control unit 10. The storage unit 20 may also function as the work memory of the control unit 10. As shown in Figure 2, the storage unit 20 may be connected to the control unit 10 by wire and / or wirelessly. The storage unit 20 may include, for example, at least one of RAM (Random Access Memory) and ROM (Read Only Memory). The storage unit 20 can be made of, for example, semiconductor memory, but is not limited to this, and can be any storage device. For example, the storage unit 20 may be a storage medium such as a memory card inserted into the electronic device 1 according to one embodiment. The storage unit 20 may also be the internal memory of the CPU used as the control unit 10, or it may be connected to the control unit 10 as a separate unit.

[0034] The communication unit 30 has the function of an interface for communicating wirelessly and / or via wired connection with, for example, external devices. The communication method performed by the communication unit 30 in one embodiment may be a wireless communication standard. For example, wireless communication standards include cellular phone communication standards such as 2G, 3G, 4G, and 5G. For example, cellular phone communication standards include LTE (Long Term Evolution), W-CDMA (Wideband Code Division Multiple Access), CDMA2000, PDC (Personal Digital Cellular), GSM (Registered Trademark) (Global System for Mobile communications), and PHS (Personal Handy-phone System). For example, wireless communication standards include WiMAX (Worldwide Interoperability for Microwave Access), IEEE802.11, WiFi, Bluetooth (Registered Trademark), IrDA (Infrared Data Association), and NFC (Near Field Communication). The communication unit 30 may include, for example, a modem whose communication method is standardized by the ITU-T (International Telecommunication Union Telecommunication Standardization Sector). The communication unit 30 can support one or more of the above communication standards.

[0035] The communication unit 30 may include, for example, an antenna for transmitting and receiving radio waves and a suitable RF unit. The communication unit 30 may communicate wirelessly with, for example, the communication unit of another electronic device via the antenna. For example, the communication unit 30 may communicate wirelessly with the terminal 100 shown in Figure 1. In this case, the communication unit 30 may communicate wirelessly with the communication unit 130 (described later) of the terminal 100. Thus, in one embodiment, the communication unit 30 has the function of communicating with the terminal 100. The communication unit 30 may also be configured as an interface such as a connector for wired connection to the outside. Since the communication unit 30 can be configured using known technologies for wireless communication, a more detailed explanation of the hardware will be omitted.

[0036] As shown in Figure 2, the communication unit 30 may be connected to the control unit 10 by wire and / or wirelessly. Various types of information received by the communication unit 30 may be supplied, for example, to the storage unit 20 and / or the control unit 10. Various types of information received by the communication unit 30 may be stored, for example, in the memory built into the control unit 10. The communication unit 30 may also transmit, for example, the processing results by the control unit 10 and / or the information stored in the storage unit 20 to an external source.

[0037] The imaging unit 40 may include an image sensor that electronically captures images, such as a digital camera. The imaging unit 40 may include an image sensor that performs photoelectric conversion, such as a CCD (Charge Coupled Device Image Sensor) or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The imaging unit 40 can capture images of the surroundings of, for example, the electronic device 1. The imaging unit 40 may capture images of the interior of the conference room MR shown in Figure 1, for example. In one embodiment, the imaging unit 40 may capture images of participants Ma, Mb, Mc, Md, Me, and Mf, etc., of a meeting held in the conference room MR shown in Figure 1.

[0038] The imaging unit 40 may convert the captured image into a signal and transmit it to the control unit 10. For this reason, the imaging unit 40 may be connected to the control unit 10 by wire and / or wirelessly. Furthermore, the signal based on the image captured by the imaging unit 40 may be supplied to the functional units of the electronic device 1, such as the storage unit 20 and the display unit 90. The imaging unit 40 is not limited to an imaging device such as a digital camera, but can be any device as long as it captures the interior of the conference room MR shown in Figure 1.

[0039] In one embodiment, the imaging unit 40 may capture images of the inside of the conference room MR as still images at predetermined intervals (e.g., 15 frames per second). In another embodiment, the imaging unit 40 may capture images of the inside of the conference room MR as a continuous video. Furthermore, the imaging unit 40 may be configured to include a fixed camera or a movable camera.

[0040] The sound collection unit 50 detects sounds or voices in the vicinity of the electronic device 1, including human voices. For example, the sound collection unit 50 may convert sounds or voices detected as air vibrations, for example by a diaphragm, into electrical signals. Specifically, the sound collection unit 50 may be configured to include an acoustic device that converts sound into electrical signals, such as any microphone. In one embodiment, the sound collection unit 50 may detect the voices of at least one of the participants Ma, Mb, Mc, Md, Me, and Mf in the conference room MR shown in Figure 1. The voices (electrical signals) detected by the sound collection unit 50 may be input to the control unit 10, for example. For this reason, the sound collection unit 50 may be connected to the control unit 10 by wire and / or wirelessly.

[0041] The sound collection unit 50 may convert the collected sound or voice into an electrical signal and supply it to the control unit 10. Alternatively, the sound collection unit 50 may supply the electrical signal (voice signal) converted from the sound or voice to a functional unit of the electronic device 1, such as the memory unit 20. The sound collection unit 50 can be any device that detects sound or voice within the conference room MR shown in Figure 1.

[0042] The amplifier 60 amplifies the sound or voice electrical signal (voice signal) supplied from the control unit 10 as appropriate and supplies it to the voice output unit 70. The amplifier 60 may include any amplifier or other device having the function of amplifying electrical signals. The amplifier 60 may amplify the sound or voice electrical signal (voice signal) according to the amplification coefficient set by the control unit 10. The amplifier 60 may be connected to the control unit 10 by wire and / or wireless.

[0043] In one embodiment, the amplifier 60 may amplify the audio signal received by the communication unit 30 from the terminal 100. Here, the audio signal received from the terminal 100 may be the audio signal of the speaker (for example, participant Mg shown in Figure 1) that the communication unit 30 receives from the speaker's terminal 100.

[0044] The audio output unit 70 outputs sound or speech by converting an electrical signal (audio signal) appropriately amplified by the amplifier 60 into sound. The audio output unit 70 may be connected to the amplifier 60 by wire and / or wirelessly. The audio output unit 70 may include a device having the function of outputting sound, such as any speaker (loudspeaker). In one embodiment, the audio output unit 70 may include a directional speaker that transmits sound in a specific direction. The audio output unit 70 may also be configured to allow the directivity of the sound to be changed.

[0045] In one embodiment, the audio output unit 70 may output the audio signal of the speaker (for example, participant Mg shown in Figure 1) amplified by the amplifier 60 as the voice of the speaker.

[0046] The direction adjustment unit 80 has the function of adjusting the direction of the sound or voice output by the audio output unit 70. The direction adjustment unit 80 may adjust the direction of the sound or voice output by the audio output unit 70 under the control of the control unit 10. For this reason, the direction adjustment unit 80 may be connected to the control unit 10 by wire and / or wireless. In one embodiment, the direction adjustment unit 80 may be configured to include a power source such as a servo motor that can change the direction of the audio output unit 70.

[0047] On the other hand, the direction adjustment unit 80 is not limited to having a function that can change the direction of the audio output unit 70. For example, the direction adjustment unit 80 may have a function that can change the orientation of the entire housing of the electronic device 1. In this case, the direction adjustment unit 80 may be configured to include a power source such as a servo motor that can change the orientation of the housing of the electronic device 1. By changing the orientation of the housing of the electronic device 1, the direction of the audio output unit 70 provided in the electronic device 1 can be changed as a result. In addition, by changing the orientation of the housing of the electronic device 1 with the direction adjustment unit 80, the orientation of the video (image) captured by the imaging unit 40 can also be changed.

[0048] Furthermore, the direction adjustment unit 80 may be provided on a pedestal or stand on which to place the housing of the electronic device 1. In this case as well, the direction adjustment unit 80 may be configured to include a power source such as a servo motor capable of changing the orientation of the housing of the electronic device 1. As described above, the direction adjustment unit 80 may have the function of changing the direction or orientation of at least one of the audio output unit 70 and the electronic device 1.

[0049] Furthermore, if the audio output unit 70 is configured to include a directional speaker, the direction adjustment unit 80 may have a function to adjust (change) the directionality of the sound or voice output by the audio output unit 70.

[0050] In one embodiment, the direction adjustment unit 80 may adjust the direction of the speaker's voice (for example, participant Mg shown in Figure 1) output by the voice output unit 70. The direction adjustment unit 80 may be configured to include any device, as long as it has the function of adjusting the direction of the sound or voice output by the voice output unit 70. The direction of the voice adjusted by the direction adjustment unit 80 may be, for example, the left-right direction (horizontal direction) and / or the up-down direction (vertical direction).

[0051] The display unit 90 may be any display device, such as a liquid crystal display (LCD), an organic electro-luminescence panel, or an inorganic electro-luminescence panel. The display unit 90 may display various types of information, such as characters, graphics, or symbols. The display unit 90 may also display various GUI objects and icon images to prompt the user to operate the electronic device 1, for example.

[0052] The various data necessary for displaying information in the display unit 90 may be supplied from, for example, the control unit 10 or the storage unit 20. For this reason, the display unit 90 may be connected to the control unit 10 or the other unit by wire and / or wirelessly. Furthermore, if the display unit 90 includes, for example, an LCD, it may be configured to include a backlight as appropriate.

[0053] In one embodiment, the display unit 90 may display an image based on a video signal transmitted from the terminal 100. The display unit 90 may display an image of, for example, participant Mg, captured by the terminal 100, as an image based on a video signal transmitted from the terminal 100. By displaying the image of participant Mg on the display unit 90 of the electronic device 1, participants Ma, Mb, Mc, Md, Me, and Mf, for example, shown in Figure 1, can visually know what participant Mg is doing, even if she is located away from the conference room MR.

[0054] The display unit 90 may, for example, display the video of participant Mg captured by terminal 100 as is. Alternatively, the display unit 90 may display an image (for example, an avatar) that characterizes participant Mg.

[0055] In one embodiment, the electronic device 1 may be a device specifically designed as described above. On the other hand, in one embodiment, the electronic device 1 may include, for example, the audio output unit 70 and the direction adjustment unit 80 among the functional units shown in Figure 2. In this case, the electronic device 1 may be connected to other electronic devices to supplement at least some of the functions of the other functional units shown in Figure 2. Here, other electronic devices may be, for example, general-purpose smartphones, tablets, phablets, notebook computers (notebook PCs or laptops), or computers (desktops).

[0056] Figure 3 is a block diagram schematically showing the configuration of the terminal 100 shown in Figure 1. Below, an example of the configuration of terminal 100 according to one embodiment will be described. As shown in Figure 1, terminal 100 may be, for example, a terminal used by participant Mg in their home RL. The electronic device 1 according to one embodiment has the function of outputting the sound input to terminal 100 when participant Mg speaks. In such a scenario, terminal 100 will also be referred to as the "speaker's terminal" as appropriate.

[0057] As shown in Figure 3, a terminal 100 according to one embodiment may include a control unit 110, a storage unit 120, a communication unit 130, an imaging unit 140, a sound collection unit 150, an audio output unit 170, and a display unit 190. In one embodiment, the terminal 100 may not include at least some of the functional units shown in Figure 3, and may include components other than the functional units shown in Figure 3.

[0058] The control unit 110 controls and / or manages the entire terminal 100, including each functional unit that constitutes the terminal 100. The control unit 110 may basically have a configuration based on the same concept as the control unit 10 shown in Figure 2, for example.

[0059] The storage unit 120 may function as a memory for storing various types of information. For example, the storage unit 120 may store programs executed in the control unit 110, and the results of processes executed in the control unit 110. The storage unit 120 may also function as a work memory for the control unit 110. As shown in Figure 3, the storage unit 120 may be connected to the control unit 110 by wire and / or wirelessly. The storage unit 120 may basically have a configuration based on the same concept as the storage unit 20 shown in Figure 2, for example.

[0060] The communication unit 130 has the function of an interface for communication by wireless and / or wired means. The communication unit 130 may communicate wirelessly with, for example, the communication unit of another electronic device via an antenna. For example, the communication unit 130 may communicate wirelessly with the electronic device 1 shown in Figure 1. In this case, the communication unit 130 may communicate wirelessly with the communication unit 30 of the electronic device 1. Thus, in one embodiment, the communication unit 130 has the function of communicating with the electronic device 1. As shown in Figure 3, the communication unit 130 may be connected to the control unit 110 by wire and / or wireless means. The communication unit 130 may basically have a configuration based on the same concept as the communication unit 30 shown in Figure 2, for example.

[0061] The imaging unit 140 may include an image sensor that electronically captures images, such as a digital camera. The imaging unit 140 may capture images of the interior of the home RL shown in Figure 1, for example. In one embodiment, the imaging unit 140 may capture images of participants Mg, for example, who are participating in a meeting from the home RL shown in Figure 1. The imaging unit 140 may convert the captured images into signals and transmit them to the control unit 110. For this purpose, the imaging unit 140 may be connected to the control unit 110 by wire and / or wirelessly. The imaging unit 140 may basically have a configuration based on the same concept as the imaging unit 40 shown in Figure 2, for example.

[0062] The sound collection unit 150 detects sounds or voices in the vicinity of the terminal 100, including human voices. For example, the sound collection unit 150 may convert sounds or voices detected as air vibrations, for example by a diaphragm, into electrical signals. Specifically, the sound collection unit 150 may be configured to include an acoustic device that converts sound into electrical signals, such as any microphone. In one embodiment, the sound collection unit 150 may detect the voice of participant Mg in the home RL shown in Figure 1. The voice (electrical signal) detected by the sound collection unit 150 may be input to the control unit 110, for example. For this reason, the sound collection unit 150 may be connected to the control unit 110 by wire and / or wirelessly. The sound collection unit 150 may basically have a configuration based on the same concept as the sound collection unit 50 shown in Figure 2, for example.

[0063] The audio output unit 170 outputs sound or speech by converting the electrical signal (audio signal) output from the control unit 110 into sound. The audio output unit 170 may be connected to the control unit 110 by wire and / or wirelessly. The audio output unit 170 may be configured to include a device having the function of outputting sound, such as any speaker (loudspeaker). In one embodiment, the audio output unit 170 may output the speech detected by the sound collection unit 50 of the electronic device 1. Here, the speech detected by the sound collection unit 50 of the electronic device 1 may be the speech of at least one of the participants Ma, Mb, Mc, Md, Me, and Mf in the conference room MR shown in Figure 1. The audio output unit 170 may basically be configured based on the same concept as the audio output unit 70 shown in Figure 2, for example.

[0064] The display unit 190 may be any display device, such as a liquid crystal display (LCD), an organic electro-luminescence panel (OLED), or an inorganic electro-luminescence panel (IEL). The display unit 190 may basically have a configuration based on the same concept as the display unit 90 shown in Figure 2, for example. Various data necessary for displaying information in the display unit 190 may be supplied from, for example, the control unit 110 or the storage unit 120. For this reason, the display unit 190 may be connected to the control unit 110, etc., by wire and / or wirelessly.

[0065] The display unit 190 may be, for example, a touchscreen display equipped with a touch panel function that detects input by touch from the participant Mg's finger or stylus.

[0066] In one embodiment, the display unit 190 may display an image based on a video signal transmitted from the electronic device 1. The display unit 190 may display images of participants Ma, Mb, Mc, Md, Me, and Mf, etc., captured by the electronic device 1 (or its imaging unit 40), as an image based on a video signal transmitted from the electronic device 1. By displaying images of participants Ma, Mb, Mc, Md, Me, and Mf, etc., on the display unit 190 of the terminal 100, participant Mg, for example, as shown in Figure 1, can visually see what participants Ma, Mb, Mc, Md, Me, and Mf, etc., are doing in a conference room MR, which is far from their home RL.

[0067] The display unit 190 may, for example, display images of participants Ma, Mb, Mc, Md, Me, and Mf captured by the electronic device 1 as they are. Alternatively, the display unit 190 may display images (e.g., avatars) that represent participants Ma, Mb, Mc, Md, Me, and Mf as characters.

[0068] In one embodiment, the terminal 100 may be a device specifically designed as described above. On the other hand, in one embodiment, the terminal 100 may include, for example, some of the functional units shown in Figure 3. In this case, the terminal 100 may be connected to other electronic devices to supplement at least some of the functions of the other functional units shown in Figure 3. Here, other electronic devices may be, for example, general-purpose smartphones, tablets, phablets, notebook computers (notebook PCs or laptops), or computers (desktops).

[0069] In particular, smartphones and laptop computers often have almost all of the functional components shown in Figure 3. For this reason, in one embodiment, terminal 100 may be a smartphone or laptop computer. In this case, terminal 100 may be a smartphone or laptop computer with an application (program) installed for coordinating with electronic device 1.

[0070] Next, the operation of the electronic device 1 and terminal 100 according to one embodiment will be described. The following description assumes a scenario where participant Mg participates in a remote meeting conducted in a conference room MR from their home RL, as shown in Figure 1.

[0071] In other words, according to one embodiment, the electronic device 1 is installed in the conference room MR and detects the voice of at least one of the participants Ma, Mb, Mc, Md, Me, and Mf. The voice detected by the electronic device 1 is transmitted to a terminal 100 installed in participant Mg's home RL. The terminal 100 outputs the voice of at least one of the participants Ma, Mb, Mc, Md, Me, and Mf that it receives from the electronic device 1. As a result, participant Mg can hear the voice of at least one of the participants Ma, Mb, Mc, Md, Me, and Mf.

[0072] On the other hand, in one embodiment, terminal 100 is installed in participant Mg's home RL and detects participant Mg's voice. The voice detected by terminal 100 is transmitted to electronic device 1 installed in conference room MR. Electronic device 1 outputs participant Mg's voice received from terminal 100. As a result, at least one of participants Ma, Mb, Mc, Md, Me, and Mf can hear participant Mg's voice.

[0073] The operation of the electronic device 1 and terminal 100 according to one embodiment will be described below, mainly from the perspective of the electronic device 1. The operation of the electronic device 1 and terminal 100 according to one embodiment may consist of roughly three stages. That is, the operation of the electronic device 1 and terminal 100 according to one embodiment is: (1) User interface (hereinafter also referred to as UI as appropriate) display phase, (2) Configuration change phase, (3) Audio output phase, It may be composed of the following. Each phase will be explained further below.

[0074] Figure 4 is a flowchart mainly illustrating the (1) UI display phase of the operation of the electronic device 1 and terminal 100 according to one embodiment. Figure 4 is a flowchart of the operation of the electronic device 1 in the (1) UI display phase according to one embodiment.

[0075] At the start of the operation shown in Figure 4, it is assumed that the electronic device 1 and the terminal 100 are connected wirelessly and / or by wire. The start of the operation shown in Figure 4 is a point in time after the electronic device 1 and the terminal 100 have been connected, and may be, for example, before the start time of the remote meeting. In other words, the start of the operation shown in Figure 4 may be, for example, during the preparation stage of the remote meeting. Through the operation shown in Figure 4, each user can check and understand the initial settings or current settings of the electronic device 1.

[0076] When the operation shown in Figure 4 begins, the control unit 10 of the electronic device 1 acquires the location of the candidate recipients (hereinafter also referred to as "candidate recipients") to whom the audio received from terminal 100 (audio of participant Mg) will be transmitted in the conference room MR (step S11). Here, the candidate recipients may be participants Ma, Mb, Mc, Md, Me, and Mf in the situation shown in Figure 1. That is, in step S11, the control unit 10 acquires the locations of the remote conference participants Ma, Mb, Mc, Md, Me, and Mf in the conference room MR.

[0077] In step S11, the location of the candidate to be transmitted may be stored in a predetermined location, for example, in a memory unit 20. For example, if the positions of chairs and other items are predetermined in a conference room MR, the control unit 10 can acquire the location of the candidate to be transmitted in advance. Even if the positions of chairs and other items are predetermined in a conference room MR, if the location of the candidate to be transmitted has not been acquired in advance, the control unit 10 may acquire the location via a communication unit 30, for example. Furthermore, if the location of the candidate to be transmitted has not been acquired in advance, the control unit 10 may detect user input at that location via an input device such as a keyboard.

[0078] Furthermore, in step S11, the control unit 10 may detect the position of the candidate to be transmitted, for example, using the imaging unit 40. The imaging unit 40 is capable of capturing images of the surroundings of the electronic device 1. Therefore, the imaging unit 40 can capture images of the meeting participants Ma, Mb, Mc, Md, Me, and Mf in the conference room MR. If the imaging unit 40 includes a movable camera, the control unit 10 may capture images of the surroundings of the electronic device 1 while changing the direction of the imaging unit 40. Also, if the direction adjustment unit 80 can change the orientation of the housing of the electronic device 1, the control unit 10 may capture images of the surroundings of the electronic device 1 while changing the orientation of the electronic device 1.

[0079] Figure 5 shows a portion of the conference room MR captured by the imaging unit 40. For example, if the direction adjustment unit 80 can change the orientation of the electronic device 1 by 360° in the horizontal direction, the imaging unit 40 can capture a 360° image of the conference room MR, as shown in Figure 5. Figure 5 may show a portion of the 360° image of the conference room MR captured by the imaging unit 40. As shown in Figure 5, the image captured by the imaging unit 40 is assumed to include conference participants Ma, Mb, and Mc.

[0080] In step S11, the control unit 10 may obtain the positions of meeting participants Ma, Mb, and Mc (candidates for information transmission) from an image as shown in Figure 5. In this case, the control unit 10 may first extract the candidates for information transmission from an image as shown in Figure 5 using existing technologies such as face recognition. Then, based on information such as the position and orientation of the electronic device 1 when the imaging unit 40 captured the candidates for information transmission, the control unit 10 may estimate the actual position of the candidates for information transmission in the conference room MR from their position in the field of view. Thus, various known techniques can be used to estimate the position of an object from an captured image.

[0081] In step S11, the control unit 10 estimates the position of participant Ma as coordinates (Xa, Ya, Za), the position of participant Mb as coordinates (Xb, Yb, Zb), and the position of participant Mc as coordinates (Xc, Yc, Zc), as shown in Figure 5. Similarly, in step S11, the control unit 10 may estimate the coordinates of the positions of participants Md, Me, and Mf based on their images, for example. For the sake of simplicity, only participants Ma, Mb, and Mc will be described below from the participants Ma, Mb, Mc, Md, Me, and Mf shown in Figure 1.

[0082] In step S11, once the locations of the candidate recipients are obtained, the control unit 10 calculates or obtains the audio level at each candidate recipient's location (step S12). That is, the control unit 10 calculates or obtains information indicating the audio level at each candidate recipient's location in the conference room MR, based on the audio of participant Mg received from terminal 100 and output from electronic device 1.

[0083] In step S12, the control unit 10 may calculate or acquire the sound level at each location based on, for example, the position of the electronic device 1, the direction of the sound output unit 70, and the position of each candidate to be transmitted. Here, the direction of the sound output unit 70 may be the direction of the directivity of the sound output from the sound output unit 70. Also, "sound level" may be various indicators that can be auditorily recognized by the candidate to be transmitted. For example, "sound level" may be the sound pressure level.

[0084] In step S12, the control unit 10 may obtain the audio level at each candidate transmission target location from data obtained in advance through demonstration experiments or the like. Such data may be stored in advance in the memory unit 20, for example, or may be obtained from outside the electronic device 1 via the communication unit 30 as needed.

[0085] On the other hand, in step S12, the control unit 10 may calculate or estimate the sound level at the location of each candidate to transmit from various data. For example, the control unit 10 may calculate or estimate the sound level at the location of each candidate to transmit based on the location and direction of the electronic device 1 (or sound output unit 70), the sound pressure of the sound output by the sound output unit 70, and the location of each candidate to transmit. Also, in step S12, the control unit 10 may calculate or estimate the sound level at each location within a predetermined range around each candidate to transmit from various data.

[0086] In step S12, once the audio level is calculated or acquired, the control unit 10 transmits information visually representing the audio level to the terminal 100 (step S13). In step S13, the control unit 10 may generate information visually representing the audio level and transmit this information from the communication unit 30 of the electronic device 1 to the communication unit 130 of the terminal 100.

[0087] Here, the information visually indicating the audio level may be, for example, information that visually suggests how audibly recognizable the voice of participant Mg output by electronic device 1 is at the location of each candidate recipient.

[0088] Figure 6 shows an example of how terminal 100 receives and displays information visually indicating the audio level transmitted in step S13 on the display unit 190. Participant Mg, who is joining the meeting from home RL, can see the information shown in Figure 6 on the display unit 190 of terminal 100 and know in advance how much of their speech will be transmitted to which participants.

[0089] Within region A12 shown in Figure 6, participants Ma and Mb are highlighted. This may indicate that within region A12, the voice of participant Mg output by electronic device 1 is audibly recognizable to participants Ma and Mb. Furthermore, within region A11 (excluding the area inside A12) shown in Figure 6, the voice of participant Mg output by electronic device 1 is barely audibly recognizable. In addition, outside region A11 shown in Figure 6, participant Mc is not highlighted. This may indicate that outside region A11, the voice of participant Mg output by electronic device 1 is almost inaudibly recognizable or completely inaudibly recognizable to participant Mc.

[0090] Figure 6 visually indicates whether each target candidate can recognize participant Mg's voice, depending on whether the images of participants Ma, Mb, and Mc are highlighted. However, in one embodiment, the way in which whether or not participant Mg's voice can be recognized is visually indicated is not limited to whether or not the image of each target candidate is highlighted.

[0091] For example, the control unit 10 may distinguish whether participant Mg's voice is recognizable by each target candidate by the intensity of the color used to display the images of participants Ma, Mb, and Mc. Alternatively, the control unit 10 may distinguish whether participant Mg's voice is recognizable by each target candidate by the transparency of the images of participants Ma, Mb, and Mc. Alternatively, the control unit 10 may distinguish whether participant Mg's voice is recognizable by each target candidate by the size of the images of participants Ma, Mb, and Mc. Furthermore, the control unit 10 may distinguish whether participant Mg's voice is recognizable by each target candidate by various modes of displaying the images of participants Ma, Mb, and Mc.

[0092] In this way, the control unit 10 transmits information to the terminal 100 that visually indicates the level of the speaker's voice at the location of a candidate target to which the speaker's voice will be transmitted. As a result, the terminal 100 can display the level of the speaker's voice at the location of a candidate target to which the speaker's voice will be transmitted on the display unit 190.

[0093] By viewing the display on the terminal 100's display unit 190 as shown in Figure 6, participant Mg can understand that their voice is audibly recognizable (i.e., audible) to participants Ma and Mb. Also, by viewing the display on the terminal 100's display unit 190 as shown in Figure 6, participant Mg can understand that their voice is not audibly recognizable (i.e., inaudible) to participant Mc.

[0094] As explained above, (1) In the UI display phase, participant Mg can determine through the UI whether or not their voice can be heard by other meeting participants.

[0095] Figure 7 is a flowchart mainly illustrating the (2) setting change phase of the operation of the electronic device 1 and terminal 100 according to one embodiment. Figure 4 is a flowchart of the operation of the electronic device 1 in the (2) setting change phase according to one embodiment.

[0096] At the point when the operation shown in Figure 7 begins, the remote conference on electronic device 1 and terminal 100 may or may not have started. Through the operation shown in Figure 7, participant Mg, who is the speaker, can select participants whose voice is audibly recognizable (i.e., participants who can hear participant Mg's voice).

[0097] When the operation shown in Figure 7 begins, the control unit 10 determines whether or not an input has been received from the terminal 100 to change the audio level (step S21). For example, suppose in step S21, participant Mg inputs using the terminal 100 that they want to change the audio level at the location of the candidate to be transmitted. Then, the terminal 100 transmits this input to the communication unit 30 of the electronic device 1 via the communication unit 130. In this case, the control unit 10 may determine that an input has been received from the terminal 100 to change the audio level.

[0098] If the terminal 100 inputs a request to change the audio level in step S21, the control unit 10 may send information to the terminal 100 for displaying a screen, for example, as shown in Figure 8, on the display unit 190 of the terminal 100. In this case, the control unit 10 may send information for displaying a screen, for example, as shown in Figure 8, from the communication unit 30 to the communication unit 130 of the terminal 100.

[0099] Figure 8 shows a screen on the display unit 190 of terminal 100 where input is made to change the audio level at the location of the candidate to be transmitted. Figure 8 shows a slider on the screen of the display unit 190 shown in Figure 6, which allows the audio level to be changed for each candidate to be transmitted. The control unit 110 may display the screen shown in Figure 8 when, for example, participant Mg makes a predetermined input on terminal 100, for example, touches the display unit 190 of terminal 100, or for example, touches any of the candidates to be transmitted on the display unit 190 of terminal 100. Participant Mg can change the audio level at the location of each candidate to be transmitted by touching the slider displayed at the location of each candidate to be transmitted.

[0100] For example, participant Mg can change the audio level at participant Ma's position by operating slider Sa, which corresponds to participant Ma displayed on the display unit 190. Participant Mg can also change the audio level at participant Mb's position by operating slider Sb, which corresponds to participant Mb displayed on the display unit 190. Furthermore, participant Mg can change the audio level at participant Mc's position by operating slider Sc, which corresponds to participant Mc displayed on the display unit 190.

[0101] In Figure 8, the slider Sa corresponding to participant Ma is at its maximum. This indicates that participant Mg's voice is auditorily recognizable (i.e., clearly audible) to participant Ma. Also in Figure 8, the slider Sb corresponding to participant Mb is at its maximum. This indicates that participant Mg's voice is auditorily recognizable (i.e., clearly audible) to participant Mb. Furthermore, in Figure 8, the slider Sc corresponding to participant Mc is at its minimum. This indicates that participant Mg's voice is auditorily unrecognizable (i.e., inaudible) to participant Mc.

[0102] If no input to change the audio level is received from the terminal 100 in step S21, the control unit 10 may wait until an input to change the audio level is received. Alternatively, if no input to change the audio level is received from the terminal 100 in step S21, the control unit 10 may terminate the operation shown in Figure 7. On the other hand, if an input to change the audio level is received from the terminal 100 in step S21, the control unit 10 calculates or obtains the amplification coefficient and the direction of the sound (step S22).

[0103] In step S22, the control unit 10 calculates or acquires the amplification coefficient and the direction of the sound to achieve the modified audio level. Here, the "amplification coefficient" may be the amplification coefficient used by the amplifier 60 to amplify the speaker's audio signal. The "direction of the sound" may be the direction of the speaker's voice adjusted by the direction adjustment unit 80. Here, the control unit 10 may acquire the amplification coefficient and the direction of the sound to achieve the modified audio level from data obtained in advance through demonstration experiments or the like. Such data may be stored in advance in the memory unit 20, for example, or acquired from outside the electronic device 1 via the communication unit 30 as needed. The control unit 10 may also calculate the amplification coefficient and the direction of the sound to achieve the modified audio level from various data. For example, the control unit 10 may calculate the amplification coefficient and the direction of the sound to achieve the modified audio level based on the position and direction of the electronic device 1 (or audio output unit 70), the sound pressure of the sound output by the audio output unit 70, and the positions of each candidate transmission target.

[0104] In step S22, once the amplification coefficient and the direction of the sound have been calculated or obtained, the control unit 10 controls at least one of the amplifier 60 and the direction adjustment unit 80 so that the amplification coefficient and the direction of the sound are realized (step S23). In step S23, if at least one of the amplification coefficient and the direction of the sound is already realized, the control unit 10 does not need to control at least one of the amplifier 60 and the direction adjustment unit 80.

[0105] In step S23, once the amplifier 60 and / or the direction adjustment unit 80 are controlled, the control unit 10 transmits information visually indicating the changed audio level to the terminal 100 (step S24). In step S13, the control unit 10 may generate information visually indicating the changed audio level and transmit this information from the communication unit 30 of the electronic device 1 to the communication unit 130 of the terminal 100. Here, the information visually indicating the audio level may be, for example, information that visually suggests how audibly recognizable the voice of participant Mg output by the electronic device 1 is at the location of each candidate transmission target, similar to the information shown in step S13 of Figure 4.

[0106] In step S24, the control unit 10 may calculate or estimate the sound level at the location of each candidate to transmit from various data. For example, the control unit 10 may calculate or estimate the sound level at the location of each candidate to transmit based on the location and direction of the electronic device 1 (or sound output unit 70), the sound pressure of the sound output by the sound output unit 70, and the location of each candidate to transmit. Alternatively, in step S24, the control unit 10 may calculate or estimate the sound level at each location within a predetermined range around each candidate to transmit from various data.

[0107] Figure 9 shows an example of how terminal 100 receives and displays information visually indicating the audio level transmitted in step S24 on the display unit 190. Participant Mg, who is joining the meeting from home RL, can see the information shown in Figure 9 on the display unit 190 of terminal 100 and know in advance how much of their speech will be transmitted to which participants.

[0108] In the region A22 shown in Figure 9, participant Ma is highlighted. This may indicate that within region A22, the voice of participant Mg output by electronic device 1 is audibly recognizable to participant Ma. Also, within region A21 (excluding A22) shown in Figure 9, the voice of participant Mg output by electronic device 1 is barely audibly recognizable. Furthermore, outside region A21 shown in Figure 9, participants Mb and Mc are not highlighted. This may indicate that outside region A21, the voice of participant Mg output by electronic device 1 is almost inaudibly recognizable or completely inaudibly recognizable to participants Mb and Mc.

[0109] As shown in Figure 8, participant Mg can operate the slider Sb corresponding to participant Mb displayed on the display unit 190 of terminal 100 to, for example, the minimum position. As a result, as shown in Figure 9, participant Mg can visually understand that the slider Sb corresponding to participant Mb displayed on the display unit 190 of terminal 100 has reached, for example, the minimum position. In other words, in the situation shown in Figure 8, participant Mg's voice was audible to participant Mb. However, in the situation shown in Figure 9, participant Mg's voice can be made inaudible to participant Mb.

[0110] For example, depending on the sound pressure and / or direction of the sound output from the sound output unit 70, it may not be possible to arbitrarily determine whether each candidate to be transmitted to is auditorily recognizable or not (i.e., whether it is audible or not). In such cases, the control unit 10 may, for example, control the slider Sb corresponding to participant Mb shown in Figure 8 so that it does not move, or control the slider Sb to return to a certain extent after it has been moved. Also, for example, when participant Mg tries to move the slider Sb corresponding to participant Mb shown in Figure 8, the control unit 10 may control the system so that it is difficult for participant Mg to perform that action. Also, for example, when participant Mg moves the slider Sb corresponding to participant Mb shown in Figure 8, the control unit 10 may control the system so that other sliders, such as the slider Sa corresponding to participant Ma, move in conjunction with it.

[0111] Figure 10 is a flowchart mainly illustrating the (3) audio output phase of the operation of the electronic device 1 and terminal 100 according to one embodiment. Figure 10 is a flowchart of the operation of the electronic device 1 in the (3) audio output phase according to one embodiment.

[0112] At the point when the operation shown in Figure 10 begins, for example, the operations shown in Figures 4 and 7 may be completed, and the remote conference using electronic device 1 and terminal 100 may have started. Through the operation shown in Figure 10, participant Mg, who is the speaker, can speak in a way that makes their voice audibly recognizable, as set in Figure 7.

[0113] When the operation shown in Figure 10 begins, the control unit 10 determines whether or not it has received audio input from terminal 100 (step S31). Here, the audio input from terminal 100 may be the input based on the voice of participant Mg detected by the sound collection unit 150 of terminal 100.

[0114] If there is an audio input from the terminal 100 in step S31, the control unit 10 controls the amplifier 60 so that the audio input is amplified according to the amplification coefficient calculated or obtained in step S22 of Figure 7 (step S32). Also in step S32, the control unit 10 controls the audio output unit 70 so that the audio amplified according to the amplification coefficient is output.

[0115] If sound is output in step S32, the control unit 10 transmits information visually indicating the level of the output sound to the terminal 100 (step S33). In step S33, the control unit 10 may generate information visually indicating the level of the sound output from the electronic device 1 and transmit this information from the communication unit 30 of the electronic device 1 to the communication unit 130 of the terminal 100. Here, the information visually indicating the level of the sound may be, for example, information that visually suggests how audibly recognizable the voice of participant Mg output by the electronic device 1 is at the location of each candidate communication target, as shown in step S24 of Figure 7.

[0116] In step S33, the control unit 10 may calculate or estimate the sound level at the location of each candidate to transmit from various data. For example, the control unit 10 may calculate or estimate the sound level at the location of each candidate to transmit based on the location and direction of the electronic device 1 (or sound output unit 70), the sound pressure of the sound output by the sound output unit 70, and the location of each candidate to transmit. Alternatively, in step S33, the control unit 10 may calculate or estimate the sound level at each location within a predetermined range around each candidate to transmit from various data.

[0117] Figure 11 shows an example of how terminal 100 receives and displays information visually indicating the audio level transmitted in step S33 on the display unit 190. Participant Mg, who is joining the meeting from home RL, can understand to what extent their speech is being transmitted to which participants by seeing the information shown in Figure 11 displayed on the display unit 190 of terminal 100.

[0118] In the region A22 shown in Figure 11, participant Ma is highlighted. This may indicate that within region A22, the voice of participant Mg output by electronic device 1 is audibly recognizable to participant Ma. Also, within region A21 (excluding A22) shown in Figure 11, the voice of participant Mg output by electronic device 1 is barely audibly recognizable. Furthermore, outside region A21 shown in Figure 11, participants Mb and Mc are not highlighted. This may indicate that outside region A21, the voice of participant Mg output by electronic device 1 is almost inaudibly recognizable or completely inaudibly recognizable to participants Mb and Mc.

[0119] Thus, in the electronic device 1 according to one embodiment, the control unit 10 sets the amplification coefficient when the amplifier 60 amplifies the speaker's voice signal. At the same time, the control unit 10 sets the direction of the speaker's voice, which is adjusted by the direction adjustment unit 80. Furthermore, the control unit 10 can change at least one of the amplification coefficient and the direction of the voice based on an input from the terminal 100 to change the level of the speaker's voice. Here, "amplification coefficient" may refer to the amplification coefficient when the amplifier 60 amplifies the speaker's voice signal. Also, "direction of voice" may refer to the direction of the speaker's voice adjusted by the direction adjustment unit 80.

[0120] In one embodiment of the electronic device 1, the audio output unit 70 may output the speaker's voice signal received by the communication unit 30 from the terminal 100 as the speaker's voice. In another embodiment, the control unit 10 may set the volume of the speaker's voice output by the audio output unit 70. In this case, the control unit 10 may change the volume of the speaker's voice output by the audio output unit 70 when the level of the speaker's voice is changed at the terminal 100. In another embodiment, the volume of the speaker's voice output by the audio output unit 70 may be changed, for example, by changing the amplification coefficient using an amplifier 60.

[0121] As described above, the electronic device 1 can output the voice of participant Mg, who is at home (RL), in the conference room (MR). Furthermore, the electronic device 1 can transmit the voice only to the recipients to whom participant Mg wishes to transmit the voice, in the conference room (MR). In other words, according to the electronic device 1 of this embodiment, participant Mg, who is the speaker, can select participants whose voice is auditorily recognizable (i.e., who can hear participant Mg's voice). Therefore, according to the electronic device 1 of this embodiment, the functionality of an electronic device that enables communication between multiple locations can be improved.

[0122] (Other embodiments) As described above, depending on the sound pressure and / or direction of the sound output from the sound output unit 70, it may not be possible to arbitrarily determine whether each candidate recipient is auditorily recognizable or not (i.e., whether they are audible or inaudible). In other words, it is conceivable that the voice of participant Mg output from the sound output unit 70 may be heard by a person among the candidate recipients whom the recipient does not want to hear (hereinafter also referred to as "non-recipient"). In such cases, the voice of participant Mg may be masked by outputting various sounds, such as noise, towards the non-recipient.

[0123] Figure 12 is a block diagram schematically showing the functional configuration of the electronic device 2 according to another embodiment. Below, an example of the configuration of the electronic device 2 according to one embodiment will be explained, focusing on the differences from the electronic device 1 described above.

[0124] Compared to the electronic device 1 shown in Figure 2, the electronic device 2 shown in Figure 12 is equipped with a second output unit 72. The second output unit 72 outputs a predetermined auditory effect. Here, the predetermined auditory effect may be various sounds or voices, such as ambient sounds, noise, the operating sounds of the electronic device 2, sound effects to attract human attention, or voices different from the voice of participant Mg.

[0125] The second output unit 72 may be located inside the housing of the electronic device 2, or it may be located outside the housing of the electronic device 2. If the second output unit 72 is located inside the housing of the electronic device 2, it may be equipped with a mechanism that can change the direction of the second output unit 72 and / or the directivity of the auditory effect output from the second output unit 72. If the second output unit 72 is located outside the housing of the electronic device 2, it may also be equipped with a mechanism that can change the direction of the second output unit 72 and / or the directivity of the auditory effect output from the second output unit 72. Furthermore, if the second output unit 72 is located outside the housing of the electronic device 2, for example, multiple second output units 72 may be provided, each corresponding to a candidate target for transmission such as participant Ma, Mb, Mc, Md, Me, and Mf. In addition, the second output unit 72 may be located in a different position from the audio output unit 70 so that it can generate different acoustic effects from the audio output unit 70.

[0126] If, for example, in step S32 of Figure 10, the voice of participant Mg output by electronic device 2 is transmitted to a non-transmission target, or there is a risk of this happening, the electronic device 2 may output a predetermined auditory effect from the second output unit 72 toward the non-transmission target. For example, in the situation shown in Figure 8, even if participant Mg lowers the slider Sb displayed on the display unit 190 of terminal 100 to a certain extent, treating participant Mb as a non-transmission target, it is conceivable that the voice level at participant Mb's position may not fall below a predetermined level. In such a case, the control unit 10 may, instead of lowering the level of the voice output from the voice output unit 70, or lower the level and also control the system so that a predetermined auditory effect is output from the second output unit.

[0127] In this case, if the control unit 10 receives an input indicating that the speaker's voice level should be changed at the terminal 100 in step S21 of Figure 7, it may control the second output unit 72 to output a predetermined auditory effect in step S23 or the like. In this way, the control unit 10 may control the auditory effect output from the second output unit 72 based on an input indicating that the speaker's voice level should be changed at the terminal 100. This reduces the risk of participant Mg's voice being transmitted to non-transmitted targets. Therefore, according to one embodiment of the electronic device 2, the functionality of the electronic device that enables communication between multiple locations can be improved.

[0128] In yet another embodiment, the second output unit 72 of the electronic device 2 may output ultrasound as a predetermined auditory effect. By outputting ultrasound to a predetermined part of the non-transmitting target or a predetermined part around the non-transmitting target, the non-transmitting target's attention may be attracted to the reflection of ultrasound, and as a result, the non-transmitting target's attention may be less likely to be directed towards the participant Mg's voice.

[0129] (Other embodiments) In the cases described above, a predetermined visual effect may be output in place of, or together with, a predetermined auditory effect.

[0130] Figure 13 is a block diagram schematically showing the functional configuration of electronic device 3 according to another embodiment. Below, an example of the configuration of electronic device 3 according to one embodiment will be explained, focusing on the differences from electronic device 1 or electronic device 2 described above.

[0131] Compared to the electronic device 1 shown in Figure 2, the electronic device 3 shown in Figure 13 includes a third output unit 93. The third output unit 93 outputs a predetermined visual effect. Here, the predetermined visual effect may be light such as an LED or a laser beam. Furthermore, the predetermined visual effect may take various forms, such as the aforementioned light emitting light for only a moment or blinking at a predetermined speed.

[0132] The third output unit 93 may be located inside the housing of the electronic device 3, or it may be located outside the housing of the electronic device 3. If the third output unit 93 is located inside the housing of the electronic device 3, it may be equipped with a mechanism that can change the direction of the third output unit 93 and / or the directivity of the visual effect output from the third output unit 93. If the third output unit 93 is located outside the housing of the electronic device 3, it may also be equipped with a mechanism that can change the direction of the third output unit 93 and / or the directivity of the visual effect output from the third output unit 93. Furthermore, if the third output unit 93 is located outside the housing of the electronic device 3, for example, multiple third output units 93 may be provided, each corresponding to a candidate target for transmission, such as participants Ma, Mb, Mc, Md, Me, and Mf.

[0133] If, for example, in step S32 of Figure 10, the voice of participant Mg output by the electronic device 3 is transmitted to a non-transmission target, or there is a risk of this happening, the electronic device 3 may output a predetermined visual effect from the third output unit 93 to the non-transmission target.

[0134] In this case, if the control unit 10 receives an input indicating that the speaker's voice level should be changed at the terminal 100 in step S21 of Figure 7, it may control the system in step S23 or later to output a predetermined auditory effect from the third output unit 93. In this way, the control unit 10 may control the auditory effect output from the third output unit 93 based on an input indicating that the speaker's voice level should be changed at the terminal 100. This reduces the risk of participant Mg's voice being transmitted to non-transmitted targets. Therefore, according to one embodiment of the electronic device 3, the functionality of the electronic device that enables communication between multiple locations can be improved.

[0135] While embodiments relating to this disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art will find it easy to make various modifications or alterations based on this disclosure. Therefore, it should be noted that these modifications or alterations are included within the scope of this disclosure. For example, the functions included in each component or step can be rearranged in a logically consistent manner, and multiple components or steps can be combined into one or divided. While embodiments relating to this disclosure have been described primarily in terms of apparatus, embodiments relating to this disclosure can also be realized as methods including steps performed by each component of the apparatus. Embodiments relating to this disclosure can also be realized as methods, programs, or storage media recording programs executed by a processor in the apparatus. These should also be understood to be included within the scope of this disclosure.

[0136] The embodiments described above are not limited to implementation as electronic device 1. For example, the embodiments described above may be implemented as a control method for a device such as electronic device 1. Furthermore, for example, the embodiments described above may be implemented as a program executed by a device such as electronic device 1. Moreover, such a program is not necessarily limited to being executed only on electronic device 1, but may also be executed on other electronic devices, such as a smartphone that interacts with electronic device 1.

[0137] Furthermore, the embodiments described above can be implemented from various viewpoints. For example, in one embodiment, it may be implemented as a system including the electronic device 1 and the terminal 100. Also, in one embodiment, it may be implemented as another electronic device (e.g., a server or control device) that can communicate with the electronic device 1 and the terminal 100. In this case, at least a part of the functions and / or operations of the electronic device 1 described in the embodiments described above may be performed by the other electronic device, such as a server. More specifically, the other electronic device, such as a server, may set the amplification coefficient (on behalf of the electronic device 1) when amplifying the audio signal that the electronic device 1 outputs as audio. In this case, the electronic device 1 can amplify the audio signal according to the amplification coefficient set by the other electronic device, such as a server, and output it as audio. Also, in one embodiment, it may be implemented as a program executed by the other electronic device, such as a server. Furthermore, in one embodiment, it may be implemented as a system including the electronic device 1, the terminal 100, and the other electronic device such as the server described above.

[0138] For example, other electronic devices mentioned above (e.g., servers or control devices) may be components like the electronic device 200 shown in Figure 14. In this case, the electronic device 200 may include, for example, a control unit 210, a storage unit 220, and a communication unit 230, as shown in Figure 15. The control unit 210 may have the same configuration and / or functions as the control unit 10 and / or control unit 110. The storage unit 220 may have the same configuration and / or functions as the storage unit 20 and / or storage unit 120. The communication unit 230 may have the same configuration and / or functions as the communication unit 30 and / or communication unit 130. Furthermore, the control unit 210 of the electronic device 200 may perform operations as shown in Figure 16, for example. The processing in each step shown in Figure 16 may be the same as or similar to the corresponding steps described in Figures 4, 7, and 10, etc. [Explanation of symbols]

[0139] 1 Electronic equipment 10 Control Unit 20 Memory section 30 Communications Department 40 Imaging Unit 50 Sound collection section 60 Amplifier 70 Audio output section 72 Second Output Section 80 Direction adjustment section 90 Display section 93 Third Output Section 100 devices 110 Control Unit 120 Storage section 130 Communications Department 140 Imaging Unit 150 Sound collection section 170 Audio output section 190 Display section 200 Electronic equipment

Claims

1. A communication unit that communicates with the speaker's terminal, The communication unit includes an audio output unit that outputs the speaker's voice signal received from the terminal as the speaker's voice, A control unit that sets the volume of the sound output by the sound output unit, Equipped with, The control unit transmits information to the terminal that visually indicates the level of the sound at the location of a candidate target to which the sound is transmitted, and the electronic device can change the volume of the sound when the level of the sound is changed in the terminal.

2. The audio output unit includes a direction adjustment unit for adjusting the direction of the sound output by the audio output unit, The electronic device according to claim 1, wherein the control unit sets the direction of the sound adjusted by the direction adjustment unit, and when the level of the sound is changed in the terminal, it can change at least one of the volume of the sound and the direction of the sound.

3. The electronic device according to claim 1 or 2, wherein the control unit can change at least one of the volume of the sound and the direction of the sound for each candidate target to which the sound is transmitted when the level of the sound is changed for each candidate target to which the sound is transmitted in the terminal.

4. The electronic device according to claim 1, wherein the control unit changes at least one of the volume of the sound and the direction of the sound so that the level of the sound is below a predetermined level at a predetermined location among the candidates for the target to which the sound is transmitted.

5. It is equipped with a second output unit that outputs a predetermined auditory effect, The electronic device according to claim 1, wherein the control unit controls the auditory effect output from the second output unit when the audio level is changed in the terminal.

6. The electronic device according to claim 5, wherein the second output unit outputs a predetermined sound wave.

7. The electronic device according to claim 5, wherein the second output unit outputs a predetermined ultrasonic wave.

8. The electronic device according to claim 5, wherein the second output unit is located at a different position from the audio output unit.

9. It includes a third output unit that outputs a predetermined visual effect, The electronic device according to claim 5, wherein the control unit controls the visual effect output from the second output unit when the audio level is changed in the terminal.

10. The electronic device according to claim 9, wherein the third output unit outputs a predetermined light.

11. In electronic devices, The steps involve communicating with the speaker's device, The steps include outputting the speaker's voice signal received from the terminal as the speaker's voice, The steps include setting the volume of the sound to be output in the output step, The steps include transmitting to the terminal information that visually indicates the level of the sound at the location of a candidate target to which the sound is to be transmitted, When the audio level is changed in the terminal, the volume of the audio can be changed. A program that executes something.

12. A system including electronic devices and speaker terminals that can communicate with each other, The aforementioned electronic device is A voice output unit that outputs the speaker's voice signal received from the terminal as the speaker's voice, A control unit that sets the volume of the sound output by the sound output unit, Equipped with, The control unit transmits information to the terminal that visually indicates the level of the sound at the candidate location of the target to which the sound is transmitted, and when the level of the sound changes in the terminal, it controls the volume of the sound to change. The aforementioned terminal is A sound collection unit that collects the voice of the aforementioned speaker, A control unit transmits the speaker's voice signal to the electronic device, receives information from the electronic device that visually indicates the level of the voice at the location of a candidate target to which the voice is to be transmitted, and controls the electronic device to transmit an input to change the level of the voice. A system equipped with these features.

13. A terminal of the speaker, and an electronic device capable of communicating with other electronic devices that output the speaker's voice signal as the speaker's voice, The system includes a control unit that sets the volume of the sound output by the other electronic device, The control unit transmits information to the terminal that visually indicates the level of the sound at the location of a candidate target to which the sound is transmitted, and when the level of the sound is changed in the terminal, it can change the volume of the sound and control other electronic devices to output the sound.

14. In electronic devices, The steps include communicating with the speaker's terminal and other electronic devices that output the speaker's voice signal as the speaker's voice, The steps include setting the volume of the sound output by the other electronic device, The steps include transmitting to the terminal information that visually indicates the level of the sound at the location of a candidate target to which the sound is to be transmitted, The steps include: changing the volume of the audio when the audio level is changed in the terminal; The steps include controlling the other electronic device to output the sound, A program that executes something.

15. A system including the speaker's terminal, electronic devices, and other electronic devices, The terminal and the electronic device are configured to communicate with the other electronic devices. The aforementioned terminal is A sound collection unit that collects the voice of the aforementioned speaker, A control unit that transmits the speaker's voice signal to the other electronic device, receives information from the other electronic device that visually indicates the level of the voice at the location of a candidate target to which the voice is to be transmitted, and controls the other electronic device to transmit an input to change the level of the voice. Equipped with, The aforementioned electronic device is A voice output unit that outputs the speaker's voice signal received from the aforementioned other electronic device as the speaker's voice, Equipped with, The aforementioned other electronic devices are The electronic device includes a control unit that sets the volume of the sound output by the electronic device, The control unit transmits information to the terminal that visually indicates the level of the sound at the candidate location of the target to which the sound is transmitted, and when the level of the sound changes in the terminal, it can change the volume of the sound and controls the electronic device to output the sound. system.

Images