Electronic device and method for performing audio communication between first space and second space

Abstract

An electronic device and a method for performing audio communication between a first space and a second space are provided. The electronic device is one of near-end devices at the first space and serves as a master device, and the other near-end device serves as a slave device. The electronic device includes a microphone, a front-end processing circuit and an audio processing circuit. The front-end processing circuit of the master device generates a master signal according to voice received by the microphone of the master device, and the front-end processing circuit of the slave device generates a slave signal according to voice received by the microphone of the slave device. The audio processing circuit of the master device generates a final signal according to the master signal and a slave signal, and the master device transmit the final signal to a far-end device at the second space via internet.

Description

Technical field

[0001] The invention relates to a voice conferencing or multi-person voice call, specifically referring to an electronic device as well as a method for conducting voice communication between a first space and a second space. Prior technology

[0002] When conducting a conference-call conference-call, participants in the same conference room can use their respective laptops to connect and record sounds without using a common conference-call speaker. Since the various laptops will receive the voices of the attendees in the same conference room with different latency due to Internet or system factors, the timing of the voices transmitted to the remote (another conference room) will be inconsistent, affecting the clarity of the voices played out when listening to the remote users. In addition, when multiple laptops in a conference room play and record sounds at the same time, the loops formed by these broadcasts / records can cause howling problems.

[0003] To avoid the above problems, users typically turn on the broadcast / record functions of only one laptop, while the rest of the laptops are turned off (e.g. set to silent mode). However, the inconsistent distance between the attendees in the conference room and the laptop being turned on makes it difficult to optimize the overall radio quality. In addition, the voices of each attendee were recorded by the same laptop, which made it impossible for remote users to determine which attendee the current voice came from through the display screen of the teleconferencing software.

[0004] Therefore, there is a need for a novel mechanism as well as related methods to improve the broadcasting / receiving quality as well as the overall user experience of conference calls conducted by multiple laptops in the same space without or less likely to bring side effects. Invention content

[0005] The purpose of this issue is to provide an electronic device for voice communication between a first space and a second space as well as a method to improve the user experience when multiple users make voice calls (e.g., improve the clarity of the sound signal, reduce unwanted interference).

[0006] At least one embodiment of the present invention provides an electronic device for voice communication between a first space and a second space. A plurality of near-end devices are located in the first space, and at least one far-end device is located in the second space, wherein the electronic device is one of the plurality of near-end devices. The electronic device includes a microphone, a front-end processing circuit, and an audio processing circuit, wherein the microphone is used to receive sound emitted from a sound source located in the first space, the front-end processing circuit is used to generate a voice signal based on the sound received by the microphone, and the audio processing circuit is used to generate a final voice signal based on the voice signal. In particular, when the electronic device is selected as a master device, each of the other near-end devices is selected as a slave device. The front-end processing circuit of the master device generates a master voice signal, and the front-end processing circuit of the slave device generates a slave voice signal. The master device obtains the slave voice signal from the slave device through a transmission interface, and the audio processing circuit in the master device generates the final voice signal based on the master voice signal and the slave voice signal, so that the master device can transmit the final voice signal to the at least one remote device through the Internet.

[0007] At least one embodiment of the present invention provides a method for conducting voice communication between a first space and a second space. A plurality of near-end devices are located in the first space, and at least one far-end device is located in the second space. The method includes: selecting one of the plurality of near-end devices as a master device and each of the other near-end devices as a slave device; receiving sound emitted from a sound source located in the first space using a microphone of the master device, and generating a master voice signal accordingly; receiving the sound emitted from the sound source located in the first space using a microphone of the slave device, and generating a slave voice signal accordingly; acquiring the slave voice signal from the slave device through a transmission interface using the master device; and generating a final voice signal based on the master voice signal and the slave voice signal using an audio processing circuit of the master device, for the master device to transmit the final voice signal to the at least one far-end device via the Internet.

[0008] The electronic device (which can be implemented in each of the plurality of near-end devices) and method provided by embodiments of the present invention configure the microphones of the various near-end devices into a microphone array to collect the voices of each user. Furthermore, the sound signals collected by these microphones can be transmitted to a near-end device selected as a master device, allowing the master device to process these sound signals so that a distant user can receive a clearer sound signal. Alternatively, the sound collected by these microphones can be transmitted to each other, allowing each device to acquire all the sound collected by the microphones, and each device can first select a master device based on the processing results of these sound signals, and then the selected master device transmits the processing results of these sound signals. In addition, embodiments of the present invention do not significantly increase additional costs. Therefore, the present invention can improve the user experience when multiple users are making voice calls without or with minimal side effects. Simple Explanation of the Diagram

[0009] Figure 1 is a schematic diagram of a plurality of near-end devices in a first space and at least one far-end device in a second space communicating via voice, according to an embodiment of the present invention. Figure 2A is a schematic diagram illustrating relevant details of communication between multiple near-end devices in a first space according to an embodiment of the present invention. Figure 2B is a schematic diagram illustrating how multiple near-end devices communicate via ultrasonic wave reception / transmission according to an embodiment of the present invention. Figure 3 is a schematic diagram of the workflow of a method for conducting voice communication between a first space and a second space according to an embodiment of the present invention. Figure 4 is a schematic diagram of multiple near-end devices exchanging information via Wi-Fi peer-to-peer transmission according to an embodiment of the present invention. Figure 5 is a schematic diagram of the workflow of multiple near-end devices communicating through a point-to-point communication protocol according to an embodiment of the present invention. Figure 6 is a schematic diagram illustrating information exchange between multiple near-end devices according to an embodiment of the present invention via the transmission / reception of ultrasonic signals. Figure 7 is a schematic diagram illustrating the workflow of multiple near-end devices communicating via the transmission / reception of ultrasonic signals according to an embodiment of the present invention. Implementation

[0010] Figure 1 is a schematic diagram illustrating voice communication between multiple near-end devices (e.g., near-end devices 110, 120, and 130) in a first space and at least one far-end device (e.g., far-end device 20) in a second space, according to an embodiment of the present invention. Near-end devices 110, 120, and 130 can communicate with each other via transmission interface IF1, near-end device 120 can communicate with the Internet 15 via transmission interface IF2, and far-end device 20 can communicate with the Internet 15 via transmission interface IF3. For example, near-end devices 110, 120, and 130 can conduct teleconferences with far-end device 20 via the Internet 15, wherein near-end devices 110, 120, and 130 can be multiple laptops located in the same conference room, and far-end device 20 can be a laptop located in another space. For example, near-end devices 110, 120 and 130 can play online games with remote device 20 via Internet 15, wherein near-end devices 110, 120 and 130 can be multiple game consoles in the same space, and remote device 20 can be a game console located in another space. In this embodiment, when the sound source 10 located in the first space emits a sound (e.g., the voice of a user speaking), this sound can be received by the near-end devices 110, 120, and 130 and a corresponding voice signal can be generated accordingly. The near-end device 110 can obtain the voice signal generated by the near-end devices 120 and 130 based on the sound emitted by the sound source 10 through the transmission interface IF1. The near-end device 120 can obtain the voice signal generated by the near-end devices 110 and 130 based on the sound emitted by the sound source 10 through the transmission interface IF1. The near-end device 130 can obtain the voice signal generated by the near-end devices 110 and 120 based on the sound emitted by the sound source 10 through the transmission interface IF1. In this embodiment, the near-end device 120 can act as a master device, such as a group owner (GO) device. The near-end device 120 can combine the voice signals generated by the near-end devices 110, 120, and 130 based on the sound emitted by the sound source 10 into a final signal, which is then transmitted to the remote device 20 via the Internet 15. It should be noted that when the near-end device 120 acts as the master device, the near-end devices 110 and 130 do not directly transmit the aforementioned voice signals to the Internet 15; that is, the voice signals collected in the first space are transmitted to the Internet 15 together by the master device (e.g., the near-end device 120).

[0011] Figure 2A is a schematic diagram illustrating the relevant details of communication between multiple proximal devices in the first space according to an embodiment of the present invention. Figure 2A is illustrated using proximal devices 110 and 120 as examples, and the implementation details involving proximal device 130 can be deduced accordingly. As shown in Figure 2A, each of the near-end devices 110 and 120 may include a microphone 210, a communication interface circuit 230, an audio processing circuit 240, a front-end processing circuit 250, a delay compensation circuit 260, a calibration circuit 270, and a network transmitter 280. The front-end processing circuit 250 is coupled to the microphone 210, the communication interface circuit 230 is coupled to the front-end processing circuit 250, the calibration circuit 270 is coupled to both the front-end processing circuit 250 and the communication interface circuit 230, the delay compensation circuit 260 is coupled to the calibration circuit 270, the audio processing circuit 240 is coupled to both the delay compensation circuit 260 and the communication interface circuit 230, and the network transmitter is coupled to the audio processing circuit 240. Specifically, the microphone 210 is used to receive sound emitted by the sound source 10 located in the first space, and the front-end processing circuit 250 is used to generate a voice signal or output a corresponding playback signal to a speaker based on the sound received by the microphone 210. In particular, the front-end processing circuit 250 can generate corresponding system delays when performing the above operations. The correction circuit 270 and the delay compensation circuit 260 can be used to compensate for the system delays and adjust the delay of the voice signal accordingly. Additionally, the audio processing circuit 240 is used to generate a final voice signal based on the voice signal, specifically based on the compensation result output by the delay compensation circuit 260, for transmission by the network transmitter 280 (e.g., to the remote device 20). When the near-end device 120 is selected as the master device, each of the other near-end devices (e.g., near-end device 110) can be selected as a slave device. For example, when the near-end device 120 shown in Figure 1 is selected as the master device, each of the near-end devices 110 and 130 can be selected as a slave device.In this embodiment, the front-end processing circuit 250 of the master device (e.g., near-end device 120) can generate a master voice signal such as voice signal VM0, and the front-end processing circuit 250 of the slave device (e.g., near-end device 110) can generate a slave voice signal such as voice signal VM1. The master device (e.g., near-end device 120) can obtain the slave voice signal from the slave device (e.g., near-end device 110) through a transmission interface IF1 (e.g., through the communication interface circuit 230 of near-end device 120 and the communication interface circuit 230 of near-end device 110). The audio processing circuit 240 in the master device (e.g., near-end device 120) can generate the final voice signal based on the master voice signal and the slave voice signal (e.g., by performing beamforming on the master voice signal and the slave voice signal). The final voice signal may be generated through audio processing methods such as forming and fusion, or by selecting either the master voice signal or the slave voice signal as the final voice signal. The master device (e.g., the network transmitter 280 of near-end device 120) may then transmit the final voice signal to the remote device 20 via the Internet 15. In some embodiments, each near-end device may obtain voice signals from other near-end devices, and each near-end device may process these voice signals to generate the final voice signal. Only the near-end device selected as the master device (e.g., near-end device 120) transmits the final voice signal to the remote device 20 via the Internet 15. For example, near-end device 110 can acquire voice signals from near-end device 120 and near-end device 130; near-end device 120 can acquire voice signals from near-end device 110 and near-end device 130; and near-end device 130 can acquire voice signals from near-end device 110 and near-end device 120. Each of near-end devices 110, 120, and 130 can generate the final voice signal based on its own voice signal and voice signals from other near-end devices. Only the near-end device selected as the master device (e.g., near-end device 120) can transmit the final voice signal to the remote device 20 via the Internet 15.

[0012] In one embodiment, when the near-end device 120 is the first among a plurality of near-end devices to connect the voice communication between the first space and the second space, the near-end device 120 can be selected as the master device. For example, when the near-end device 120 shown in Figure 1 is the first among near-end devices 110, 120, and 130 to connect to the online teleconference (meaning that near-end devices 110 and 130 are not yet connected to the online teleconference), the near-end device 120 can begin broadcasting ultrasonic messages (e.g., messages carrying conference-related information such as conference identification codes) through its internal speaker. At this time, the near-end device 120 does not receive ultrasonic messages from other near-end devices (e.g., ultrasonic messages broadcast in response to the near-end device 120's broadcast), therefore the near-end device 120 can be set as the master device. When other near-end devices such as 110 and 130 connect to the online teleconference after near-end device 120 (or enter the first space after near-end device 120), near-end devices 110 and 130 can respectively send ultrasonic signals (e.g., messages carrying conference-related information such as conference identification codes). Since near-end device 120 has been set as the master device, near-end device 120 can respectively respond to the ultrasonic signals sent by near-end devices 110 and 130 with corresponding ultrasonic response messages. Furthermore, near-end devices 110 and 130 can receive ultrasonic messages from the master device such as near-end device 120 and be set as the slave device accordingly. Through the broadcasting and communication of the aforementioned ultrasound, the near-end device 120 can determine whether other near-end devices exist in the same space (e.g., the first space). In addition, the speakers of each of the near-end devices 110, 120, and 130 can send a test ultrasound signal, and the microphone 210 of the main device (e.g., near-end device 120) can receive the test ultrasound signal sent by each of the near-end devices 110, 120, and 130 to detect the distance and angle between the main device and each of the plurality of near-end devices (e.g., between near-end devices 110 and 120 or between near-end devices 130 and 120), thereby establishing the topology of a microphone array and a speaker array formed by the near-end devices 110, 120, and 130. For example, when the near-end device 120 enters the first space and attempts to receive ultrasonic signals, if the near-end device 120 fails to receive ultrasonic signals from other near-end devices, the near-end device 120 can be automatically set as the master device and then begin periodically playing ultrasonic signals. Therefore, when near-end devices 110 and 130 enter the first space and attempt to receive ultrasonic signals, near-end devices 110 and 130 can receive ultrasonic signals from near-end device 120, and thus near-end devices 110 and 130 can be automatically set as slave devices.

[0013] In one embodiment, after all near-end devices 110, 120, and 130 are connected to an online teleconference, each of the near-end devices 110, 120, and 130 can test the stability of each near-end device's connection to the Internet. When a particular near-end device (e.g., near-end device 120) is the near-end device among the near-end devices 110, 120, and 130 with the best network stability, near-end device 120 can be selected as the master device, while other near-end devices such as 110 and 130 can be set as the slave devices.

[0014] In some embodiments, after all the near-end devices 110, 120 and 130 are connected to the online teleconference, each of the near-end devices 110, 120 and 130 may select the near-end device (e.g. near-end device 120) located in the optimal position (e.g., the topological center of the speaker array and / or the topological center of the microphone array) as the master device, based on the topology of the speaker array and / or the microphone array of the near-end devices 110, 120 and 130, while the other near-end devices (e.g. near-end devices 110 and 130) may be set as the slave devices.

[0015] In some embodiments, the proximal device acting as the master device can be dynamically switched. Assuming that initially, near-end device 120 is set as the master device and other near-end devices such as 110 and 130 are set as slave devices, when the audio processing circuit 240 of near-end device 120 determines that the difference between a slave audio indicator (e.g., the intensity or quality of the sound received by the microphone 210 of near-end device 110) of the slave voice signal generated by a specific near-end device (e.g., near-end device 110) and a master audio indicator (e.g., the intensity or quality of the sound received by the microphone 210 of near-end device 120) of the master voice signal generated by near-end device 120 reaches a predetermined threshold (e.g., the intensity of the sound received by the microphone 210 of near-end device 110 is greater than the intensity of the sound received by the microphone 210 of near-end device 120 and the difference between the two is greater than the predetermined threshold), near-end device 120 can be switched to the slave device, and the specific near-end device (e.g., near-end device 110) can be switched to the master device through the transmission interface IF1. After the switch is completed, the work performed by the master device is instead performed by the near-end device 110 (for example, by transmitting the final voice signal to the Internet 15 through the transmission interface IF2).

[0016] It should be noted that the switching between the master and slave devices will temporarily interrupt data transmission on the transmission interface IF2. If someone is speaking in the first space while the near-end device 120 is acting as the master device, and the near-end device 110 is immediately switched to the master device, the speaker of the near-end device 110 will play the sound of the speech in the first space before the switch as the far-end sound due to network latency immediately after switching to the master device. Therefore, when the near-end device 120 needs to release the position of the master device, a delay is required after the release before notifying the near-end device 110 to take over the position of the master device. In some embodiments, to avoid the sound in the first space being interrupted due to the switching of the master device, the master device can determine whether the current time is a suitable switching point based on whether the sound intensity in the first space is low enough. For example, to prevent the remote device 20 from being affected by the switching between the master and slave devices when transmitting remote voice signals to the master device (e.g., near-end device 120) among the near-end devices 110, 120, and 130, the audio processing circuit 240 of the near-end device 120 determines whether the slave audio indicator (e.g., the intensity or quality of the sound received by the microphone 210 of the near-end device 110) of the slave voice signal generated by a specific near-end device (e.g., near-end device 110) is different from the master audio indicator (e.g., the sound received by the microphone 210 of the near-end device 120) of the master voice signal generated by the near-end device 120. When the difference between the intensity or sound quality of the sound received by the microphone 210 of the near-end device 110 reaches a predetermined threshold (for example, when the intensity of the sound received by the microphone 210 of the near-end device 120 is greater than the intensity of the sound received by the microphone 210 of the near-end device 120 and the difference between the two is greater than the predetermined threshold), and the intensity of a remote voice signal transmitted by the far-end device 20 to the master device (e.g., the near-end device 120) is greater than an intensity threshold (indicating that the remote user is speaking and the volume is loud enough to allow the far-end device 20 to transmit a valid remote voice signal to the master device), the near-end device 120 can continue to act as the master device without performing the above-mentioned switching until the intensity of the remote voice signal does not exceed the intensity threshold.

[0017] In this embodiment, the communication between the communication interface circuit 230 in the near-end device 120 and the communication interface circuit 230 in the near-end device 110 can be an example of the transmission interface IF1. The communication interface circuit 230 in the near-end device 120 and the communication interface circuit 230 in the near-end device 110 can be a Wi-Fi point-to-point transmission interface such as a Wi-Fi direct transmission interface. The slave device (e.g., the near-end device 110) can perform Wi-Fi point-to-point transmission with the master device (e.g., the near-end device 120) through the Wi-Fi direct transmission interface to transmit the slave voice signal, such as the voice signal VM1, to the master device (e.g., the near-end device 120).

[0018] Figure 2B is a schematic diagram illustrating communication between multiple near-end devices, such as 110 and 120, via ultrasonic wave reception / transmission according to an embodiment of the present invention. Each of the communication interface circuits 230 of near-end device 110 and 120 may include a boost circuit 222U, a speaker 222SP, a microphone 222MIC, and a down-conversion circuit 222D. In some embodiments, the microphone 210 of near-end device 110 shown in Figure 2A and the microphone 222MIC in the communication interface circuit 230 of near-end device 110 shown in Figure 2B may be the same microphone, and the microphone 210 of near-end device 120 shown in Figure 2A and the microphone 222MIC in the communication interface circuit 230 of near-end device 120 shown in Figure 2B may be the same microphone. In some embodiments, the microphone 210 of the near-end device 110 shown in Figure 2A and the microphone 222MIC in the communication interface circuit 230 of the near-end device 110 shown in Figure 2B may be different microphones, and the microphone 210 of the near-end device 120 shown in Figure 2A and the microphone 222MIC in the communication interface circuit 230 of the near-end device 120 shown in Figure 2B may be different microphones. In some embodiments, the slave device (e.g., near-end device 110) may use its internal upsampling circuit 222U to carry a slave-end voice signal, such as voice signal VM1, onto an ultrasonic signal so that the speaker 222SP of the slave device (e.g., near-end device 110) plays the ultrasonic signal, such as voice signal VM3. The microphone 222MIC of the master device (e.g., near-end device 120) receives the ultrasonic signal played by the speaker 222SP of the slave device (e.g., near-end device 110), allowing the master device (e.g., near-end device 120) to obtain the slave-end voice signal carried on the ultrasonic signal through its internal downsampling circuit 222D. That is, the transmission / reception of ultrasonic waves can be used as another example of the transmission interface IF1. The transmission of signals / information from near-end device 120 to near-end device 110 can be deduced in the same way, and will not be elaborated here for the sake of simplicity.

[0019] Figure 3 is a schematic diagram of the workflow of a method for conducting voice communication between a first space and a second space according to an embodiment of the present invention, wherein the workflow can be executed by a system consisting of the near-end devices 110, 120, and 130 shown in Figure 1. It should be noted that the workflow shown in Figure 3 is for illustrative purposes only and is not intended to limit the invention. For example, one or more steps can be added, deleted, or modified in the workflow shown in Figure 3. Furthermore, these steps do not necessarily have to be performed in the exact order shown in Figure 3 if the same result can be obtained.

[0020] In step S300, the system may select one of the proximal devices 110, 120 and 130 as the master device (e.g., select proximal device 120 as the master device) and each of the other proximal devices (e.g., proximal devices 110 and 130) as the slave device.

[0021] In step S310, the system can use the microphone 210 of the master device (e.g., the near-end device 120) to receive the sound emitted by the sound source 10 located in the first space, and generate a master voice signal accordingly.

[0022] In step S320, the system can use the microphones of devices 110 and 130 to receive the sound emitted by the sound source located in the first space, and generate a slave voice signal accordingly (for example, device 110 generates a first slave voice signal based on the sound received by microphone 210 of device 110, and device 130 generates a second slave voice signal based on the sound received by microphone 210 of device 130).

[0023] In step S330, the system can use the master device (e.g., the near-end device 120) to obtain the slave voice signal from the devices 110 and 130 through the transmission interface IF1 (e.g., obtaining the first slave voice signal from the device 110 and the second slave voice signal from the device 130).

[0024] In step S340, the system can use the audio processing circuit 240 of the master device (e.g., the near-end device 120) to generate a final audio signal based on the master voice signal and the slave voice signal (e.g., based on the master voice signal, the first slave voice signal, and the second slave voice signal), so that the master device (e.g., the near-end device 120) can transmit the final audio signal to the remote device 20 through the transmission interface IF2 and the Internet 15.

[0025] Figure 4 is a schematic diagram illustrating the information exchange between multiple near-end devices, such as 110 and 120, according to an embodiment of the present invention via Wi-Fi point-to-point transmission, such as Wi-Fi direct transmission. For ease of explanation, it is assumed that near-end device 120 is selected as the master device and near-end device 110 is selected as the slave device, wherein the Internet 15 and the remote device 20 are not shown in Figure 4 for simplicity. Before a Wi-Fi direct connection is established between the master device (e.g., near-end device 120) and the slave device (e.g., near-end device 110), the speaker 220 of the near-end device 120 (which may be an example of the speaker 222SP of the near-end device 120 shown in Figure 2B) can emit a master ultrasonic signal carrying the master Wi-Fi point-to-point information of the near-end device 120 (e.g., upsampling the master Wi-Fi point-to-point information to the ultrasonic band to generate the master ultrasonic signal), so that the near-end device 110 can obtain the master Wi-Fi point-to-point information of the near-end device 120 through the microphone 210 of the near-end device 110 (e.g., downsampling the master ultrasonic signal received by the microphone 210 to obtain the master Wi-Fi point-to-point information). Additionally, the speaker 220 of the near-end device 110 (which may be an example of speaker 222SP of the near-end device 110 shown in Figure 2B) can emit a slave ultrasonic signal carrying the slave Wi-Fi peer-to-peer information of the near-end device 110 (e.g., upsampling the slave Wi-Fi peer-to-peer information to the ultrasonic band to generate the slave ultrasonic signal), so that the near-end device 120 can obtain the slave Wi-Fi peer-to-peer information of the near-end device 110 through the microphone 210 of the near-end device 120 (e.g., downsampling the slave ultrasonic signal received by the microphone 210 to obtain the slave Wi-Fi peer-to-peer information). That is, the near-end devices 110 and 120 can exchange their Wi-Fi peer-to-peer information with each other via ultrasound, thereby establishing a direct Wi-Fi connection between the near-end devices 110 and 120. In particular, the near-end device 120 can perform a direct Wi-Fi connection with the near-end device 110 based on the master-end Wi-Fi peer-to-peer information and the slave-end Wi-Fi peer-to-peer information to obtain the slave-end voice signal from the near-end device 110. In some embodiments, the slave-end ultrasonic signal and the master-end ultrasonic signal may be emitted in the same frequency band but at different times. In some embodiments, the slave-end ultrasonic signal and the master-end ultrasonic signal may be in different frequency bands.

[0026] After the Wi-Fi Direct connection between near-end devices 110 and 120 is established, near-end device 120 can obtain the slave voice signal and the timestamp information corresponding to the slave voice signal from near-end device 110 through Wi-Fi Direct connection (which can be taken as an example of the transmission interface IF1 shown in Figure 1). The delay compensation circuit 260 in near-end device 120 is used to control the delay of the master voice signal and the slave voice signal according to the timestamp information to generate a delay compensation result. The audio processing circuit 240 in near-end device 120 can generate the final voice signal according to the delay compensation result. In detail, both the microphone 210 of the near-end device 110 and the microphone 210 of the near-end device 120 can receive the sound emitted by the sound source 10. The front-end processing circuit 250 of the near-end device 120 can generate the master voice signal and the corresponding timestamp information (such as a timestamp recording the system delay of the front-end processing circuit 250 of the near-end device 120, such as T2) based on the sound received by the microphone 210 of the near-end device 120. The front-end processing circuit 250 of the near-end device 110 can generate the slave voice signal and the corresponding timestamp information (such as a timestamp recording the system delay of the front-end processing circuit 250 of the near-end device 110, such as T1) based on the sound received by the microphone 210 of the near-end device 110. In particular, the communication interface circuit 230 can transmit the slave voice signal and the corresponding timestamp information to the communication interface circuit 230 in the near-end device 120 via Wi-Fi direct transmission, so that the delay compensation circuit 260 of the near-end device 120 can obtain the slave voice signal and the corresponding timestamp information through the communication interface circuit 230 of the near-end device 120, and the delay compensation circuit 260 of the near-end device 120 can obtain the master voice signal and the corresponding timestamp information from the front-end processing circuit 250 of the near-end device 120. Therefore, the delay compensation circuit 260 can compare the timestamp information corresponding to the master voice signal and the timestamp information corresponding to the slave voice signal (e.g., comparing the system delay T1 of the front-end processing circuit 250 of the near-end device 110 and the system delay T2 of the front-end processing circuit 250 of the near-end device 120), and control the delay of the master voice signal and / or the slave voice signal based on the comparison result (e.g., the difference between system delays T1 and T2), thereby generating the delay compensation result. In particular, by comparing the timestamp information corresponding to the master voice signal and the timestamp information corresponding to the slave voice signal (e.g., comparing the system delay T1 of the front-end processing circuit 250 of the near-end device 110 and the system delay T2 of the front-end processing circuit 250 of the near-end device 120), the comparison result can indicate the overall result of the transmission delay caused by Wi-Fi Direct and the system delays T1 and T2.

[0027] Figure 5 is a schematic diagram illustrating the workflow of multiple near-end devices, such as 110 and 120, communicating via a point-to-point communication protocol (e.g., the aforementioned Wi-Fi Direct transmission) according to an embodiment of the present invention. This workflow can be executed by a system comprised of the near-end devices 110 and 120 shown in Figure 4. It should be noted that the workflow shown in Figure 5 is for illustrative purposes only and is not intended to limit the scope of the invention. For example, one or more steps can be added, deleted, or modified in the workflow shown in Figure 5. Furthermore, these steps do not necessarily have to be performed in the exact order shown in Figure 5 if the same result can be obtained.

[0028] In step S510, the system can use the microphone 210 of the master device (e.g., the near-end device 120) to receive the sound emitted by the sound source 10 located in the first space, and generate the master voice signal and master delay information (e.g., timestamp information recording system delay T2).

[0029] In step S520, the system can use the microphone 210 of the slave device (e.g., the near-end device 110) to receive the sound emitted by the sound source 10 located in the first space, and generate the slave voice signal and slave delay information (e.g., timestamp information recording system delay T1).

[0030] In step S530, the system can use the master device (e.g., the near-end device 120) to obtain the slave voice signal and the slave delay information from the slave device (e.g., the near-end device 110) through a point-to-point communication protocol (e.g., the Wi-Fi Direct transmission described above).

[0031] In step S540, the system can use the delay compensation circuit 260 of the master device (e.g., the near-end device 120) to compare the master delay information and the slave delay information, and perform delay compensation on the master voice signal or the slave voice signal accordingly.

[0032] In step S550, the system (e.g., the audio processing circuit 240 in the near-end device 120) can generate the final voice signal based on the result of delay compensation, so that the master device (e.g., the near-end device 120) can transmit the final voice signal to the remote device 20 through the transmission interface IF2 and the Internet 15.

[0033] Figure 6 is a schematic diagram illustrating information exchange between multiple near-end devices, such as 110 and 120, according to an embodiment of the present invention via the transmission / reception of ultrasonic signals. For ease of explanation, it is assumed that near-end device 120 is selected as the master device and near-end device 110 is selected as the slave device, wherein the Internet 15 and the remote device 20 are not shown in Figure 4 for simplicity. In this embodiment, near-end device 110 can carry the slave voice signal on a slave ultrasonic signal U11 (e.g., up-convert the slave voice signal to the ultrasonic band to generate the slave ultrasonic signal U11) and emit the slave ultrasonic signal U11 through the speaker 220 of near-end device 110, so that the microphone 210 of near-end device 120 receives the slave ultrasonic signal U11, thereby obtaining the slave voice signal carried on the slave ultrasonic signal U11 (e.g., down-convert the slave ultrasonic signal U11 received by microphone 210 to obtain the slave voice signal). Additionally, the near-end device 120 can carry the master voice signal on a master ultrasonic signal U21 (e.g., upsampling the master voice signal to the ultrasonic band to generate the master ultrasonic signal U21) and emit the master ultrasonic signal U21 through the speaker 220 of the near-end device 120, so that the microphone 210 of the near-end device 110 receives the master ultrasonic signal U21, thereby obtaining the slave voice signal carried on the master ultrasonic signal U21 (e.g., downsampling the master ultrasonic signal U21 received by the microphone 210 to obtain the master voice signal). In some embodiments, the slave ultrasonic signal U11 emitted by the speaker 220 of the near-end device 110 and the master ultrasonic signal U21 emitted by the speaker 220 of the near-end device 120 may be emitted in the same frequency band but at different times. In some embodiments, the slave ultrasonic signal U11 emitted by the speaker 220 of the proximal device 110 and the master ultrasonic signal U21 emitted by the speaker 220 of the proximal device 120 may be in different frequency bands.

[0034] In addition, the system delay T1 caused by the front-end processing circuit 250 in the near-end device 110, the system delay T2 caused by the front-end processing circuit 250 in the near-end device 120, and the transmission delay caused by the near-end devices 110 and 120 transmitting signals via ultrasonic signals (e.g., slave ultrasonic signal U11 or master ultrasonic signal U21) can also be estimated by transmitting / receiving ultrasonic signals. For example, the speaker 220 of the near-end device 110 can emit a near-end test ultrasonic signal U10 based on a near-end test signal (e.g., upsampling the near-end test signal to the ultrasonic band to generate the near-end test ultrasonic signal U10), and the microphone 210 of the near-end device 110 can receive the near-end test ultrasonic signal U10 and generate a near-end feedback signal accordingly (e.g., downsampling the near-end test ultrasonic signal U10 received by the microphone 210 to generate the near-end feedback signal). The near-end device 110 can generate near-end delay information based on the near-end test signal and the near-end feedback signal (e.g., comparing the near-end test signal and the near-end feedback signal to obtain the information of the system delay T1), so that the near-end device 120 can obtain the near-end delay information from the near-end device 110 through the transmission interface IF1 (e.g., through the above-mentioned Wi-Fi direct connection or ultrasonic transmission / reception). Additionally, the speaker 220 of the near-end device 120 can emit a master-end test ultrasonic signal U20 based on a master-end test signal (e.g., upsampling the master-end test signal to the ultrasonic band to generate the master-end test ultrasonic signal U20), and the microphone 210 of the near-end device 120 can receive the master-end test ultrasonic signal U20 and generate a master-end feedback signal accordingly (e.g., downsampling the master-end test ultrasonic signal U20 received by the microphone 210 to generate the master-end feedback signal). The near-end device 120 can generate master-end delay information based on the master-end test signal and the master-end feedback signal (e.g., comparing the master-end test signal and the master-end feedback signal to obtain the information of the system delay T2), so that the near-end device 110 can obtain the master-end delay information from the near-end device 120 through the transmission interface IF1 (e.g., through the above-mentioned Wi-Fi direct connection or ultrasonic transmission / reception). When the proximal device 120 acts as the master device, the delay compensation circuit 260 of the proximal device 120 is used to control the delay of the master voice signal and the slave voice signal according to the master delay information and the slave delay information to generate a delay compensation result, and the audio processing circuit 240 in the proximal device 120 can generate the final voice signal according to the delay compensation result.

[0035] The difference between the embodiment in Figure 6 and the embodiment in Figure 4 lies in the method of information transmission between near-end device 110 and near-end device 120 (i.e., the implementation of transmission interface IF1) and the method of estimating system information T1 and T2. All other implementation details are the same and will not be repeated here. Furthermore, the implementation of transmission interface IF1 in the embodiment of Figure 4 is not limited to Wi-Fi Direct; for example, it can be implemented using ultrasonic wave transmission / reception. The implementation of the method for estimating system information T1 and T2 in the embodiment of Figure 4 is not limited to recording timestamp information; for example, it can be estimated using ultrasonic wave transmission / reception. Similarly, the implementation of transmission interface IF1 in the embodiment of Figure 6 is not limited to ultrasonic wave transmission / reception; for example, it can be implemented using Wi-Fi Direct. The implementation of the method for estimating system information T1 and T2 in the embodiment of Figure 6 is not limited to estimating using ultrasonic wave transmission / reception; for example, it can be implemented using timestamp information.

[0036] Figure 7 is a schematic diagram illustrating the workflow of multiple near-end devices, such as 110 and 120, communicating via the transmission / reception of ultrasonic signals according to an embodiment of the present invention. This workflow can be performed by a system comprised of the near-end devices 110 and 120 shown in Figure 6. It should be noted that the workflow shown in Figure 7 is for illustrative purposes only and is not intended to limit the invention. For example, one or more steps can be added, deleted, or modified in the workflow shown in Figure 7. Furthermore, these steps do not necessarily have to be performed in the exact order shown in Figure 7 if the same result can be obtained.

[0037] In step S710, the system can use the microphone 210 of the master device (e.g., the near-end device 120) to receive the sound emitted by the sound source 10 located in the first space, and generate the master voice signal accordingly. Additionally, the system can use the microphone 210 of the master device (e.g., the near-end device 120) to receive its own emitted ultrasonic signal (e.g., the master test ultrasonic signal U20 shown in Figure 6), and generate master delay information for the master device accordingly.

[0038] In step S720, the system can use the microphone 210 of the slave device (e.g., the near-end device 110) to receive the sound emitted by the sound source 10 located in the first space, and generate the slave voice signal accordingly. Additionally, the system can use the microphone 210 of the slave device (e.g., the near-end device 110) to receive its own emitted ultrasonic signal (e.g., the slave test ultrasonic signal U10 shown in Figure 6), and generate slave delay information for the slave device accordingly.

[0039] In step S731, the system may use a slave device (e.g., near-end device 110) to carry the slave voice signal and the slave delay information in a slave ultrasonic signal (e.g., upsampling the slave voice signal and the slave delay information to the ultrasonic band to generate the slave ultrasonic signal), and emit the slave ultrasonic signal through the speaker 220 of the slave device (e.g., near-end device 110).

[0040] In step S732, the system can use the microphone 210 of the master device (e.g., the near-end device 120) to receive the slave ultrasonic signal from the slave device (e.g., the near-end device 110) to obtain the slave voice signal and the slave delay information (e.g., down-converting the slave ultrasonic signal received by the microphone 210 to obtain the slave voice signal and the slave delay information).

[0041] In step S740, the system can use the delay compensation circuit 260 of the master device (e.g., the near-end device 120) to compare the master delay information and the slave delay information, and perform delay compensation on the master voice signal or the slave voice signal accordingly.

[0042] In step S750, the system (e.g., the audio processing circuit 240 in the near-end device 120) can generate the final voice signal based on the result of delay compensation, so that the master device (e.g., the near-end device 120) can transmit the final voice signal to the remote device 20 through the transmission interface IF2 and the Internet 15.

[0043] Since the microphones 210 of multiple near-end devices such as 110, 120, and 130 in the same space can be turned on simultaneously, these microphones can be regarded as a distributed microphone array. The signals received by the microphones 210 of different near-end devices may have different intensities, directions, sound qualities, and other patterns or characteristics. Therefore, the main device (e.g., near-end device 120) can classify speakers based on these characteristics to record the speech of different users during the teleconference based on the speaker classification results. It should be noted that each of the near-end devices 110, 120, and 130 can obtain the signals received by the microphones 210 of other near-end devices, and each can process these signals and transmit the processed results to other near-end devices, but the present invention is not limited thereto.

[0044] In summary, embodiments of the present invention provide a method for multiple near-end devices in the same space to transmit / receive voice signals received by their respective microphones via Wi-Fi Direct or ultrasonic transmission / reception. This allows these voice signals to be collected by a single near-end device (e.g., a specific near-end device acting as a master device). These voice signals can be treated as voice signals collected by a microphone array for processing by the master device (e.g., beamforming, fusion), and the processing results are transmitted over the Internet to a remote location to optimize the quality of the transmitted voice signal. Furthermore, embodiments of the present invention do not significantly increase additional costs. Therefore, the present invention can improve the user experience when multiple users are making voice calls without or with minimal side effects. The above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made in accordance with the claims of the present invention shall be covered by the present invention.

[0045] 10: Sound Source 15: Internet 20: Remote device 110, 120, 130: Proximal devices IF1, IF2, IF3: Transmission Interface 210: Microphone 220: Speaker 230: Communication Interface Circuit 240: Audio processing circuit 250: Front-end processing circuit 260: Delay Compensation Circuit 270: Correction circuit 280: Network transmitter 222U: Up-frequency circuit 222D: Frequency reduction circuit 222SP: Speaker 222MIC: Microphone VM0, VM1, VM3: Voice signals U10, U11, U20, U21: Ultrasonic signals S300~S340, S510~S550, S710~S750: Steps

Claims

1. An electronic device for voice communication between a first space and a second space, wherein a plurality of near-end devices are located in the first space and at least one far-end device is located in the second space, the electronic device being one of the plurality of near-end devices, and the electronic device comprising: a microphone for receiving sound emitted from a sound source located in the first space; a front-end processing circuit for generating a voice signal based on the sound received by the microphone; and an audio processing circuit for generating a final voice signal based on the voice signal; wherein when the electronic device is selected as a master device, each of the other near-end devices is selected as a slave device, the front-end processing circuit of the master device generates a master voice signal, the front-end processing circuit of the slave device generates a slave voice signal, the master device obtains the slave voice signal from the slave device through a transmission interface, and the audio processing circuit in the master device generates the final voice signal based on the master voice signal and the slave voice signal, for the master device to transmit the final voice signal to the at least one far-end device via the Internet; When the audio processing circuit of the electronic device determines that the difference between a slave audio index of the slave voice signal generated by a specific near-end device among the plurality of near-end devices and a master audio index of the master voice signal generated by the electronic device reaches a predetermined threshold, the electronic device is switched to the slave device, and the specific near-end device is switched to the master device.

2. The electronic device as described in claim 1, wherein when the electronic device is the first of the plurality of proximal devices for connecting the first space and the second space for voice communication, the electronic device is selected as the master device.

3. The electronic device as described in claim 1, wherein the main device performs speaker classification based on the pattern of signals received by the microphone of each of the plurality of near-end devices, so as to record the speech of different users according to the speaker classification results.

4. An electronic device for voice communication between a first space and a second space, wherein a plurality of near-end devices are located in the first space and at least one far-end device is located in the second space, the electronic device being one of the plurality of near-end devices, and the electronic device comprising: a microphone for receiving sound emitted from a sound source located in the first space; a front-end processing circuit for generating a voice signal based on the sound received by the microphone; and an audio processing circuit for generating a final voice signal based on the voice signal; wherein when the electronic device is selected as a master device, each of the other near-end devices is selected as a slave device, the front-end processing circuit of the master device generates a master voice signal, the front-end processing circuit of the slave device generates a slave voice signal, the master device obtains the slave voice signal from the slave device through a transmission interface, and the audio processing circuit in the master device generates the final voice signal based on the master voice signal and the slave voice signal, so that the master device can transmit the final voice signal to the at least one far-end device via the Internet; When the audio processing circuit of the electronic device determines that the difference between a slave audio index of the slave voice signal generated by a specific near-end device among the plurality of near-end devices and a master audio index of the master voice signal generated by the electronic device reaches a switching threshold, and the strength of a far-end voice signal transmitted by at least one far-end device to the master device is greater than a strength threshold, the electronic device continues to act as the master device.

5. The electronic device as described in claim 4, wherein the electronic device further includes a speaker, the speaker of the main device emitting a master ultrasonic signal carrying master-end Wi-Fi peer-to-peer information of the main device, so that the slave device can obtain the master-end Wi-Fi peer-to-peer information of the main device through the microphone of the slave device, the speaker of the slave device emitting a slave ultrasonic signal carrying slave-end Wi-Fi peer-to-peer information of the slave device, so that the main device can obtain the slave-end Wi-Fi peer-to-peer information of the slave device through the microphone of the main device, and the main device performs Wi-Fi peer-to-peer transmission with the slave device based on the master-end Wi-Fi peer-to-peer information and the slave-end Wi-Fi peer-to-peer information to obtain the slave-end voice signal from the slave device.

6. The electronic device as described in claim 4, wherein the electronic device further includes a speaker, the speaker of each of the plurality of near-end devices transmitting a test ultrasonic signal, and the microphone of the main device receiving the test ultrasonic signal transmitted by each of the plurality of near-end devices to detect the distance and angle between the main device and each of the plurality of near-end devices, thereby establishing a topology of a microphone array and a speaker array formed by the plurality of near-end devices.

7. The electronic device as described in claim 4, wherein the electronic device further comprises: a delay compensation circuit, wherein the master device obtains the slave voice signal and timestamp information corresponding to the slave voice signal from the slave device through the transmission interface, and the delay compensation circuit in the master device is used to control the delay of the master voice signal and the slave voice signal according to the timestamp information to generate a delay compensation result, and the audio processing circuit in the master device generates the final voice signal according to the delay compensation result.

8. The electronic device as described in claim 4, wherein the electronic device further includes a speaker, and the slave device carries the slave voice signal on a slave ultrasonic signal and emits the slave ultrasonic signal through the speaker of the slave device, such that the microphone of the master device receives the slave ultrasonic signal, thereby obtaining the slave voice signal carried on the slave ultrasonic signal.

9. A method for conducting voice communication between a first space and a second space, wherein a plurality of near-end devices are located in the first space and at least one far-end device is located in the second space, and the method comprises: selecting one of the plurality of near-end devices as a master device and each of the other near-end devices as a slave device; receiving sound emitted from a sound source located in the first space using a microphone of the master device, and generating a master voice signal thereon; receiving the sound emitted from the sound source located in the first space using a microphone of the slave device, and generating a slave voice signal thereon; acquiring the slave voice signal from the slave device through a transmission interface using the master device; and generating a final voice signal based on the master voice signal and the slave voice signal using an audio processing circuit of the master device, for the master device to transmit the final voice signal to the at least one far-end device via the Internet; When the audio processing circuit of the electronic device determines that the difference between a slave audio index of the slave voice signal generated by a specific near-end device among the plurality of near-end devices and a master audio index of the master voice signal generated by the electronic device reaches a predetermined threshold, the electronic device is switched to the slave device, and the specific near-end device is switched to the master device.

Images