Equipment management method and electronic equipment

Abstract

The invention discloses an equipment management method and electronic equipment. The method comprises the following steps: acquiring voice feature data of at least two pieces of electronic equipment in a first space; the first electronic device is determined based on the voice feature data of the at least two electronic devices, namely, the device used by the speaker is determined, and the first electronic device is controlled to send the picked sound signal to other electronic devices. And controlling a second electronic device other than the device used by the speaker not to send the picked sound signal to other electronic devices, and controlling an audio output device of the second electronic device not to output the sound signal of the first space picked in the current time window. In this way, echo signals in the remote call process can be reduced. By adopting the method and the device, the echo signals in the remote call process can be reduced, the echo signals comprise the echo signal of the home-end space and the echo signal of the far-end space, and the definition of the remote call is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a device management method and electronic equipment. Background Art

[0002] Remote calls are widely used in scenarios such as voice chat and video chat among multiple people. When there are multiple electronic devices participating in the remote call in the local space, when the user of one of the electronic devices speaks, the sound will be picked up by the electronic device and other electronic devices. Due to the difference in the distance between these electronic devices and the speaker, the time when each electronic device picks up the sound signal is inconsistent, and the time when the electronic device in the remote space receives the sound signal picked up by each electronic device is also different, resulting in an echo signal in the remote space. In addition, the electronic device used by the speaker transmits the picked up sound signal to other electronic devices in the local space, and other electronic devices play the speaker's sound signal. However, since the speaker's voice will also be transmitted through the air in the local space, the user in the local space will hear multiple identical sound signals with time differences, resulting in an echo signal in the local space. Therefore, how to solve the echo signal during the remote call is an urgent problem to be solved. Summary of the invention

[0003] The present application provides a device management method and an electronic device, which can reduce echo signals during a remote call, wherein the echo signals include echo signals in a local space and echo signals in a remote space, thereby improving the clarity of the remote call.

[0004] In a first aspect, an embodiment of the present application provides a device management method, the method comprising: acquiring voice feature data of at least two electronic devices in a first space; the voice feature data of each electronic device is obtained by extracting features of a sound signal picked up by each electronic device within a current time window; determining a first electronic device based on the voice feature data of at least two electronic devices, and controlling the first electronic device to send the picked up sound signal to other electronic devices, and controlling a second electronic device not to send the picked up sound signal to other electronic devices; the second electronic device includes electronic devices other than the first electronic device in the first space, and the other electronic devices include electronic devices in the first space and electronic devices in the second space; controlling an audio output device of the second electronic device not to output the sound signal of the first space picked up within the current time window.

[0005] It can be seen that by determining the first electronic device in the first space, the first electronic device can be controlled to send the picked-up sound signal to other electronic devices, so that the sound signal output by the electronic device in the second space is clearer. By controlling the second electronic device not to send the picked-up sound signal to other electronic devices, the problem of sending multiple sound signals with time differences to the electronic devices in the second space causing the existence of echo signals in the second space can be avoided. In addition, by controlling the audio output device of the second electronic device not to output the sound signal of the first space picked up in the current time window, no echo signal will be generated in the first space, thereby solving the echo problem in the first space during the remote call and improving the clarity of the remote call.

[0006] In combination with the first aspect, in a possible implementation, determining a first electronic device based on voice feature data of at least two electronic devices includes: performing feature comparison on audio feature information included in the voice feature data of at least two electronic devices to determine a first similarity between sound signals picked up by the at least two electronic devices; and when the first similarity is greater than or equal to a first similarity threshold, comparing time indication information included in the voice feature data of at least two electronic devices to determine a first electronic device that picks up the sound signal earliest among the at least two electronic devices.

[0007] It can be seen that by comparing the first similarity between the sound signals picked up by at least two electronic devices and comparing the time indication information of at least two devices, it can be determined whether the sound signals picked up by at least two electronic devices are the speech content of the same speaker, and it can be determined which electronic device among multiple electronic devices picked up the sound signal earliest, so that the electronic device is used as the first electronic device. Since the first electronic device picks up the sound signal earliest and is closest to the speaker, the sound signal picked up is the clearest and has the lowest delay. After being sent to other electronic devices, the sound signal output by the electronic device in the second space is also clearer, which can improve the clarity of the remote call.

[0008] In combination with the first aspect, in a possible implementation, feature comparison is performed on audio feature information included in voice feature data of at least two electronic devices to determine a first similarity between sound signals picked up by at least two electronic devices, including: inputting the audio feature information respectively included in the voice feature data of at least two electronic devices into a voice processing model for feature comparison to determine the first similarity between the sound signals picked up by at least two electronic devices; wherein the voice processing model is obtained by training an initial model based on similarity labels between at least two sample sound signals and sample similarities between at least two sample sound signals, and the sample similarity is obtained by feature comparison of sample audio feature information of at least two sample sound signals based on the initial model.

[0009] It can be seen that by using a speech processing model to perform feature comparison on the audio feature information included in the speech feature data of at least two electronic devices, since the speech processing model is obtained by training the initial model, the accuracy of the feature comparison can be improved. Using the speech processing model for feature comparison can also improve the efficiency of the feature comparison.

[0010] In combination with the first aspect, in a possible implementation, controlling the audio output device of the second electronic device not to output the sound signal of the first space picked up within the current time window includes: obtaining audio feature information of the sound signal received by the second electronic device; performing feature comparison on the audio feature information of the sound signal received by the second electronic device and the audio feature information included in the voice feature data of the first electronic device, and determining a second similarity between the sound signal received by the second electronic device and the sound signal picked up by the first electronic device; when the second similarity is greater than or equal to a second similarity threshold, determining that the sound signal received by the second electronic device is the sound signal of the first space picked up within the current time window, and controlling the audio output device of the second electronic device not to output the received sound signal.

[0011] It can be seen that by combining the audio feature information of the sound signal received by the second electronic device with the audio feature information included in the voice feature data of the first electronic device to perform feature comparison to determine the second similarity between the two, it can be determined whether the sound signal received by the second electronic device is the sound signal of the first space picked up in the current time window. If so, the audio output device of the second electronic device is controlled not to output the received sound signal. In this way, only the speech of the speaker transmitted through the air is transmitted in the first space, and the sound signal transmitted through the network is not output, which can avoid the generation of echo signals in the first space.

[0012] In combination with the first aspect, in a possible implementation, the method also includes: when the second similarity is less than a second similarity threshold, determining that the sound signal received by the second electronic device is a sound signal of the second space picked up within the current time window, and controlling the audio output device of the second electronic device to output the received sound signal.

[0013] It can be seen that when it is determined that the sound signal received by the second electronic device is a sound signal of the second space picked up within the current time window, the audio output device of the second electronic device is controlled to output the received sound signal, and the sound signal picked up by the electronic device in the second space is output in the first space to avoid the problem of missing the speech sound of the second space such as the far-end space in the remote call.

[0014] In combination with the first aspect, in a possible implementation, controlling a first electronic device to send a picked-up sound signal to other electronic devices, and controlling a second electronic device not to send a picked-up sound signal to other electronic devices, includes: sending a first control instruction to the first electronic device; the first control instruction is used to control the first electronic device to send the picked-up sound signal to other electronic devices; sending a second control instruction to the second electronic device; the second control instruction is used to control the second electronic device not to send the picked-up sound signal to other electronic devices.

[0015] It can be seen that by sending a first control instruction for controlling the first electronic device to send the picked-up sound signal to other electronic devices to the first electronic device, the first electronic device can send the picked-up sound signal to other electronic devices. By sending a second control instruction for controlling the second electronic device not to send the picked-up sound signal to other electronic devices to the second electronic device, the second electronic device may not send the picked-up sound signal to other electronic devices. By sending different control instructions to the first electronic device and the second electronic device respectively, the accuracy of controlling each electronic device can be improved, and it is avoided that the first electronic device and the second electronic device both send the picked-up sound signal to other electronic devices, resulting in an echo signal in the second space.

[0016] In combination with the first aspect, in a possible implementation, sending a first control instruction to a first electronic device includes: obtaining an initial working state of the first electronic device within a current time window; the initial working state is used to indicate whether to send a picked-up sound signal to other electronic devices; and sending the first control instruction to the first electronic device when the initial working state of the first electronic device indicates not to send a picked-up sound signal to other electronic devices.

[0017] It can be seen that by obtaining the initial working state of the first electronic device in the current time window, it is possible to determine which control instruction to send to the first electronic device, thereby improving the control accuracy of the first electronic device and reducing the echo problem in the remote call.

[0018] In combination with the first aspect, in a possible implementation, the method also includes: receiving a sound signal sent by a third electronic device and picked up by a third electronic device; when the third electronic device is an electronic device in the second space, controlling an audio output device of the second electronic device to output the sound signal picked up by the third electronic device.

[0019] It can be seen that when receiving the sound signal picked up by the electronic device in the second space, the audio output device of the second electronic device is controlled to output the sound signal picked up by the third electronic device to avoid the problem of sound leakage in the far-end space during the remote call.

[0020] In combination with the first aspect, in a possible implementation, obtaining voice feature data of at least two electronic devices in the first space includes: establishing a communication connection with at least two electronic devices in the first space; and receiving voice feature data respectively transmitted by the at least two electronic devices based on the communication connection.

[0021] It can be seen that by establishing a communication connection with multiple electronic devices, voice feature data sent by each electronic device can be received based on the communication connection, and then the first electronic device can be determined based on the voice feature data of each electronic device, thereby improving the accuracy of determining the first electronic device.

[0022] In a second aspect, the present application provides an electronic device, comprising: one or more processors and one or more memories; the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions. When the one or more processors execute the computer instructions, the electronic device executes the device management method in any possible implementation of the first aspect above.

[0023] In a third aspect, the present application provides a chip system, which includes a processor and an interface, and the processor and the interface are coupled; the interface is used to receive or output signals, and the processor is used to execute code instructions to execute the device management method in any possible implementation of the first aspect above.

[0024] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the device management method in any possible implementation of the first aspect above.

[0025] In a fifth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the device management method in any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1a Schematic diagram 1 of an application scenario provided for an embodiment of the present application; Figure 1b An example of an application scenario provided in the present application Figure 2 ; Figure 2 A flowchart of a device management method provided in an embodiment of the present application is shown in FIG1 ; Figure 3 An example of an application scenario provided in the present application Figure 3 ; Figure 4A schematic diagram of an application scenario for processing echoes in a far-end space provided in an embodiment of the present application; Figure 5 A schematic diagram of a device management method provided in an embodiment of the present application Figure 2 ; Figure 6 A schematic diagram of an application scenario for processing echoes in a local space provided in an embodiment of the present application; Figure 7 A flowchart of the interaction between modules of an electronic device provided in an embodiment of the present application; Figure 8 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application; Fig. 9 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0028] It should be understood that the terms "first", "second", etc. in the specification, claims and drawings of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0029] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0030] In order to facilitate understanding of the solution provided by the embodiment of the present application, the following introduces the relevant concepts involved in the embodiment of the present application: 1. Remote call Remote calls refer to scenarios where electronic devices in multiple different spaces participate in the same call, such as participating in the same voice chat, video chat, conference, live broadcast room, game, etc. Electronic devices in multiple different spaces participate in the same remote call, which can realize the interaction between electronic devices in multiple different spaces. During the remote call, two devices on the electronic devices participating in the remote call are mainly used to pick up and output the sound signals in the space, namely the audio input device and the audio output device.

[0031] 2. Audio Input Device The audio input device is a device for picking up and transmitting sound signals in the space where the electronic device is located. The audio input device may refer to, for example, a microphone (Microphone, Mic), which is also called a microphone or a mouthpiece.

[0032] 3. Audio Output Device The audio output device is a device for outputting the sound signal received by the electronic device. The audio output device may be, for example, a loud speaker, which may also be called a speaker.

[0033] 4. Echo Signals in Far Space In the case where there are multiple electronic devices participating in a remote call in the same space, such as the local space, when the user of one of the electronic devices speaks, the sound will be picked up by the electronic device and by other electronic devices in the local space. Since the distance between each electronic device in the local space and the speaker is different, the time when each electronic device picks up the sound signal is inconsistent. And since each electronic device in the local space will send the sound signal to all electronic devices participating in the remote call through the network after picking up the sound signal, including the electronic device in the remote space and all electronic devices in the local space, there is also a difference in the time when the electronic device in the remote space receives the sound signal picked up by each electronic device in the local space. The electronic device in the remote space outputs the received multiple sound signals with time differences, which will generate an echo signal in the remote space.

[0034] 5. Echo signal of local space The electronic device used by the speaker transmits the picked-up sound signal to each electronic device in the local space through the network, and the audio output device of each electronic device in the local space outputs the received sound signal of the speaker. Since the speaker's voice will also propagate through the air in the local space, and the speed of air propagation is greater than the speed of network propagation, there will be multiple identical sound signals with time differences in the local space, thus generating an echo signal in the local space.

[0035] The following describes the application scenarios of the embodiments of the present application: See also Figure 1a , Figure 1a Schematic diagram 1 of an application scenario provided in an embodiment of the present application. Figure 1a As shown, Figure 1a For example, it can refer to a scene in which multiple participants in a group office use their own office equipment to join the same meeting, or it can refer to a scene in which multiple game players in the same room use their own game equipment to join the same game, and multiple game players in the room access the call system; or in a live broadcast scene, the anchor and multiple viewers join the same live broadcast room in the same room for voice interaction, etc. Figure 1a In the remote call process shown, taking a collective office scenario as an example, the different spaces where multiple electronic devices participating in the remote call are located can be divided into remote spaces and local spaces, and the electronic devices in the remote space and the local space all participate in the same remote call, such as conference A. When there are two electronic devices participating in the remote call sharing the same space such as space 1, for example, when there are multiple electronic devices in the local space, the user of electronic device 1 among the multiple electronic devices can be defined as participant 1, and the user of electronic device 2 can be defined as participant 2. The remote space is explained by taking an electronic device participating in a remote conference as an example, and the user of the electronic device in the remote space is defined as a remote participant. Optionally, there may be multiple electronic devices participating in conference A in the remote space, and the present application does not limit the number of electronic devices in the remote space. Sound signals can be transmitted between the remote space and the local space through the network.

[0036] See also Figure 1b , Figure 1b An example of an application scenario provided in the present application Figure 2 .like Figure 1b As shown, during the remote call, for example, when participant 2 is speaking, Figure 1b As shown in ① in FIG, the audio input device (such as a microphone) on the electronic device 2 used by the participant 2 picks up the voice signal of the participant 2, and as shown in FIG. Figure 1bAs shown in ②, the audio input device (such as a microphone) on the electronic device 1 of participant 1 can pick up the sound signal of the local space, and thus also pick up the sound signal of participant 2. Since the distances between electronic device 1 and electronic device 2 and participant 2 are different, the time when electronic device 2 picks up the sound signal of participant 2 is different from the time when electronic device 1 picks up the sound signal of participant 2. When electronic device 1 and electronic device 2 respectively send the picked-up sound signals to the electronic device in the remote space through the network, the remote electronic device outputs two sound signals with a time difference through the audio output device (such as a speaker), thereby generating an echo signal in the remote space, reducing the call clarity and quality in the remote space.

[0037] Therefore, in order to solve the problem that the echo generated in the far-end space affects the clarity of the remote call, by determining the device where the speaker is located in the current time window in the local space, for example, when participant 2 is speaking, the electronic device 2 used by participant 2 is controlled to send the picked-up sound signal to other electronic devices, and the electronic device 1 is controlled not to send the picked-up sound signal to other electronic devices, then the far-end electronic device only receives the sound signal sent by the electronic device 2, thereby solving the echo signal problem in the far-end space and improving the call clarity and quality in the far-end space during the remote call. Among them, the above-mentioned other electronic devices at least include electronic devices in the far-end space, such as Figure 1b Remote electronic equipment in.

[0038] To solve the echo problem in the local space during a remote call, such as Figure 1b As shown in ③, the voice signal of participant 2 is transmitted through the air to the ears of each participant in the local space, such as the ears of participant 1 and participant 2. Figure 1b As shown in ④, the voice signal of participant 2 is transmitted by electronic device 2 used by participant 2 through Figure 1b The audio signal of participant 2 is picked up by the method in ① and sent to electronic device 1 through the network. The audio output device (such as a speaker) of electronic device 1 outputs the audio signal of participant 2 sent by electronic device 2, and the audio signal of participant 2 is played out through the speaker in the local space. Figure 1b ③ and Figure 1b The sound signals output by the two methods in ④ have a time difference, so each participant in the local space will hear two echo signals with a time difference.

[0039] Therefore, in order to solve the echo problem generated in the local space, when the device where the speaker is located in the current time window in the local space is determined, for example, when participant 2 speaks, electronic device 2 picks up the sound signal and sends it to electronic device 1. By controlling the audio output device of electronic device 1 not to output the sound signal of participant 2 picked up by electronic device 2, only the speech of participant 2 transmitted through the air is played in the local space. Therefore, the echo signal problem in the local space can be solved, the call clarity in the local space during the remote call is improved, and the remote call experience is improved.

[0040] Next, combine Figure 2 For a detailed description of the specific process of the device management method provided in the embodiment of the present application, please refer to Figure 2 , Figure 2 A flow chart of a device management method provided in an embodiment of the present application is shown in FIG. Figure 2 It is mainly to solve the echo signal problem in the far-end space during long-distance calls, such as Figure 2 As shown, the method may include but is not limited to the following steps: S101, acquiring voice feature data of at least two electronic devices in a first space.

[0041] The device management method in the embodiment of the present application can be executed by any electronic device in any space, or by the audio management module on any electronic device, or can also be executed by a chip or chip system with the function of the audio management module. There are multiple electronic devices in the space that share the space. Among them, the electronic device can refer to a mobile phone, a tablet computer, a laptop computer, a PDA, a mobile internet device (MID), a wearable device, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, a personal digital assistant (PDA), etc., and the embodiment of the present application is not limited to this.

[0042] Among them, the first space can refer to the local space. There are multiple electronic devices in the first space that share the first space. Sharing the first space means that multiple electronic devices participate in the same remote call in the first space, and each electronic device can pick up the sound signal in the first space and transmit it to other electronic devices participating in the remote call.

[0043] Optionally, the embodiments of the present application may include two kinds of spaces, namely, a first space and a second space, and the second space may refer to a space other than the first space, for example, if the first space is a local space, then the second space is a remote space. The first space and the second space are different spaces, and the first space and the second space are relative, that is, when the first space is a remote space, then the second space is a local space. The first space and the second space are located in different positions, and the electronic device in the first space and the electronic device in the second space can transmit the sound signals picked up in their respective spaces through the network. For example, an electronic device in the first space receives a sound signal transmitted by an electronic device in the second space through the network, and the received sound signal can be output by each electronic device in the first space. An electronic device in the second space receives a sound signal transmitted by an electronic device in the first space through the network, and the received sound signal can be output by each electronic device in the second space, thereby realizing a remote call between electronic devices in two different spaces.

[0044] It is understandable that, taking the first space as the local space as an example, the number of second spaces can be multiple. When there are multiple different spaces, any one space can be selected as the first space, and the remaining spaces can be called second spaces.

[0045] In the embodiment of the present application, an electronic device is selected from a first space having a shared space of multiple electronic devices, and the device management method is executed by the selected electronic device as an example. For the convenience of description, the electronic device selected in the first space is referred to as the designated device of the first space. By selecting the designated device from the first space, other electronic devices in the first space can send their respective data to the designated device for analysis and processing, thereby determining the device used by the speaker, and then performing different device management on the device used by the speaker and other devices respectively, to solve the echo signal problem in remote calls.

[0046] Optionally, the designated device of the first space can be selected by one or more combinations of the following methods: determining an electronic device in the first space whose time of joining the remote call is greater than a time threshold as the designated device of the first space; or, receiving the first selection instruction for any electronic device in the first space, and determining the selected any electronic device as the designated device of the first space; or, obtaining performance level indication information for each electronic device in the first space, and determining an electronic device whose performance level indicated by the performance level indication information is greater than a level threshold as the designated device in the first space; or, receiving voting information for each electronic device in the first space, and determining an electronic device whose number of votes indicated by the voting information is greater than a quantity threshold as the designated device in the first space. Wherein, the higher the performance of the electronic device is, the higher the efficiency of the electronic device in processing subsequent data is.

[0047] In the embodiment of the present application, the accuracy of the selection of the designated device can be improved by selecting a designated device from multiple electronic devices in the first space through one or more combinations of the above methods. And each electronic device in the first space can determine to send its own status information such as status information of the audio input device and the audio output device, voice feature data and other information to the designated device, and then the designated device can analyze each electronic device in the first space, thereby performing different types of control on each electronic device, improving the accuracy of the control of each electronic device, and thus reducing the echo signal in the remote call.

[0048] Among them, after determining the designated device of the first space, each electronic device in the first space can transmit voice feature data to the designated device of the first space respectively, then the designated device of the first space can obtain its own voice feature data, and determine the voice feature data of at least two electronic devices in the first space based on its own voice feature data and the received voice feature data transmitted by each electronic device in the first space, so as to determine the device used by the speaker in the first space based on the voice feature data of at least two electronic devices in the first space, and then perform different controls on the device used by the speaker and other devices. The device used by the speaker can refer to the electronic device closest to the speaker, such as the electronic device held by the speaker, or the device that uses the speaker's account information to participate in the meeting. When the speaker switches, the device used by the speaker will also switch accordingly. For example, when participant 1 speaks, the electronic device used by the speaker is electronic device 1, and when participant 2 speaks, the electronic device used by the speaker is electronic device 2.

[0049] Optionally, before obtaining the voice feature data of at least two electronic devices in the first space, the designated device of the first space can respectively establish communication connections with the at least two electronic devices in the first space, and receive the voice feature data respectively transmitted by the at least two electronic devices based on the communication connections.

[0050] The communication connection establishment method may include but is not limited to Bluetooth (BT), wireless local area networks (WLAN) (such as Wi-Fi networks) or other local shared device interconnection methods. This application does not limit the communication connection establishment method between electronic devices. By establishing a communication connection with at least two electronic devices in the first space, the efficiency of each electronic device sending voice feature data to the designated device in the first space can be improved, and the network transmission channel in the remote call will not be occupied, which can improve the stability of the remote call.

[0051] Among them, the voice feature data of each electronic device is obtained by extracting features from the sound signal picked up by each electronic device in the current time window. In the embodiment of the present application, the sound signal picked up in each time window can be used as an example to illustrate, and the device used by the speaker in each time window can be determined, so as to perform different types of management on each electronic device in the first space within the time window in combination with the device used by the speaker.

[0052] Optionally, before each electronic device sends its own voice feature data to the designated device of the first space, it can pre-extract features from the sound signal picked up in the current time window to obtain voice feature data. Each electronic device extracts features from its own sound signal and then transmits them to the designated device of the first space. The designated device of the first space can directly analyze the voice feature data of at least two electronic devices to determine the electronic device used by the speaker in the first space. There is no need for each electronic device to transmit the sound signal itself and then perform feature extraction by the designated device of the first space, which can improve the efficiency of voice feature data transmission.

[0053] Among them, the voice feature data may include, for example, but not limited to, one or more of the Mel-scale frequency cepstral coefficients (MFCC) features, energy, chroma, phase and other features of the sound signal. Among them, energy is related to the amplitude of the sound. The larger the amplitude, the higher the energy of the sound, which is usually expressed as the loudness of the sound. Chroma is used to describe the timbre. The chroma is determined by the spectral structure of the sound. Different people will produce different timbres when speaking. Phase represents the position of the waveform on the time axis. The change of phase will affect the waveform of the sound, thereby affecting the characteristics of the sound. Optionally, the voice feature data of the electronic device can be expressed in the form of a feature vector or a feature matrix. The feature vector or feature matrix represented by the voice feature data of each electronic device is different. By calculating the similarity distance between the feature vectors or feature matrices of two electronic devices, the similarity between the voice feature data of the two electronic devices can be determined, thereby determining the similarity between the sound signals picked up by the two electronic devices, thereby determining whether the sound content of the sound signals picked up by the two electronic devices is the same. For example, the similarity between two voice feature data can be determined by a similarity distance calculation formula. The similarity distance calculation formula may include but is not limited to cosine distance, Euclidean distance, Manhattan distance, Hamming distance, Chebyshev distance, and the like.

[0054] Optionally, each electronic device can extract features from the sound signal picked up in the current time window to obtain audio feature information such as MFCC features in the following manner: sample the sound signal picked up by each electronic device at a preset sampling period, and transform the continuous sound signal into a discretized sound signal. The sampling period can be a period determined according to the Nyquist sampling theorem; then filter the discretized sound signal through a digital filter with a transfer function of H(Z)=1-αZ-1 to increase the high-frequency resolution of the sound signal, α is a pre-emphasis coefficient, α is greater than 0.9 and less than 1; then, the discrete sound signal can be framed using a window function to obtain multiple voice frames, that is, N voice frames are obtained here, wherein the window function can be any one of a rectangular window, a Hamming window or a Hanning window. Further, each voice frame is subjected to a fast Fourier transform to obtain the energy distribution of each voice frame on the spectrum, that is, each voice frame after frame division and windowing is subjected to a fast Fourier transform to obtain the spectrum of each frame, and the spectrum of the sound signal is squared to obtain the energy spectrum of the sound signal. Next, the energy spectrum is filtered through a set of Mel-scale triangular filter groups, and finally the logarithmic energy of the output of each triangular filter group is calculated and brought into the discrete cosine transform (DCT) to obtain the MFCC features of the sound signal.

[0055] Optionally, each electronic device in the first space may also send the sound signal picked up in the current time window to a designated device in the first space, and the selected designated device extracts the features of the sound signal sent by each electronic device to obtain the voice feature data of each electronic device. Since each electronic device does not need to extract the voice feature data of the sound signal picked up in the current time window separately, the computing power of each electronic device can be reduced.

[0056] See also Figure 3 , Figure 3 An example of an application scenario provided in the present application Figure 3 ,like Figure 3 As shown, by adding an audio management module to each electronic device in the local space, for example, the audio management module can be added to the audio path on each electronic device, such as the audio processing object (APO), and a communication connection is established between the audio management modules on each electronic device to realize the transmission of voice feature data in the audio management module. By transmitting the voice feature data of each electronic device to the audio management module of the designated device in the first space, the audio management module of the designated device in the first space can analyze and determine the similarity between the sound signals picked up by at least two electronic devices in the first space, and perform time delay synchronization analysis on the sound signals picked up by at least two electronic devices to determine the first electronic device that picked up the sound signal the earliest.

[0057] Optionally, if there are multiple electronic devices sharing a space in the remote space, an audio management module may be added to the audio path of each electronic device in the remote space, and a communication connection may be established between the audio management modules on each electronic device, so as to realize the similarity analysis between the voice feature data transmission and the sound signal in the audio management module on each electronic device in the remote space and determine the first electronic device that picks up the sound signal earliest in the remote space.

[0058] S102, performing feature comparison on audio feature information included in voice feature data of at least two electronic devices to determine a first similarity between sound signals picked up by the at least two electronic devices and a first electronic device that picks up the sound signal earliest among the at least two electronic devices.

[0059] Here, by selecting a designated device of the first space from a plurality of electronic devices in the first space, the designated device of the first space can be analyzed based on the voice feature data of at least two electronic devices to determine the electronic device used by the speaker in the first space, that is, the first electronic device. Since the first electronic device is closest to the speaker, the sound signal picked up by the first electronic device is the clearest, and the speed of picking up the sound signal is the fastest, that is, the delay in the sound signal is the lowest, so the first electronic device can be subsequently controlled to send the picked-up sound signal to other electronic devices, and the second electronic device other than the first electronic device can be controlled not to send the picked-up sound signal to other electronic devices. It can be avoided that both the first electronic device and the second electronic device send sound signals with time difference and different clarity to other electronic devices, resulting in the generation of echo signals.

[0060] Optionally, the electronic device in the first space may determine the first electronic device by analyzing the voice feature data of at least two electronic devices in the first space. For example, the designated device in the first space may perform feature comparison on the voice feature data of at least two electronic devices to determine the first similarity between the sound signals picked up by the at least two electronic devices and the first electronic device that picked up the sound signal earliest among the at least two electronic devices.

[0061] The speech feature data may include audio feature information and time indication information. The audio feature information may reflect the sound characteristics in the sound signal. For example, the audio feature information may include one or more of the energy, chromaticity, loudness, etc. of the sound signal. By comparing the similarity between the audio feature information of two sound signals, the similarity between the sound contents of the two sound signals may be determined, thereby determining whether the contents of the two sound signals are the same, that is, determining whether the two sound signals are obtained by different electronic devices picking up the voice of the same speaker.

[0062] Among them, the time delay between the two sound signals can be reflected by comparing the time indication information between the two sound signals. The lower the time delay between the two sound signals, the closer the electronic device that picks up the two sound signals is to the speaker. Since the distances between the multiple electronic devices in the first space and the speaker are different, there is a time delay difference between each electronic device picking up the speaker's voice, that is, the speed at which the sound signal is picked up by the farther electronic device is slower than the speed at which the sound signal is picked up by the closer electronic device, so that there is a time delay difference between the sound signals picked up by different electronic devices. By comparing the time delay difference between any two sound signals, the first electronic device that picks up the sound signal the earliest can be determined, that is, the first electronic device closest to the speaker. For example, the time indication information may refer to the phase in the sound signal or other information that can be used to indicate the time delay between sound signals.

[0063] Optionally, for example, the time indication information may also refer to the time when each electronic device picks up the speaker's voice signal in the first space. Before obtaining the time indication information in the voice feature data of each electronic device, the device time on each electronic device may be calibrated to make the time on each electronic device consistent, thereby improving the accuracy of subsequent comparison of the time when each electronic device picks up the speaker's voice signal in the first space.

[0064] In the embodiment of the present application, by comparing the audio feature information in the voice feature data of at least two electronic devices, it can be determined whether the contents of the sound signals picked up by the at least two electronic devices are the same or similar. If so, it is considered that the sound signals picked up by the at least two electronic devices are obtained by different electronic devices for the speech sound of the same speaker in the current time window. Further, by comparing the time indication information included in the voice feature data of at least two electronic devices, the time delay difference between the sound signals picked up by the at least two electronic devices can be determined, thereby determining the first electronic device that picked up the sound signal the earliest.

[0065] Optionally, the designated device in the first space may use a speech processing model to perform feature comparison on speech feature data of at least two electronic devices, and determine a first similarity between the sound signals picked up by the at least two electronic devices and the first electronic device that picked up the sound signal earliest among the at least two electronic devices. The speech processing model may include, but is not limited to, a deep neural network model (DNN), a recurrent neural network model (RNN), a residual neural network model (ResNet), a convolutional neural network model (CNN), and the like.

[0066] Optionally, before using the speech processing model, a large number of sample sound signals can be obtained in advance to train the initial model to obtain the speech processing model, so that the trained speech processing model can have the ability to perform feature comparison on the speech feature data of at least two electronic devices, determine the first similarity between the sound signals picked up by at least two electronic devices, and determine the first electronic device that picks up the sound signal earliest among at least two electronic devices, so that when the speech processing model is used for feature comparison later, the efficiency and accuracy of feature comparison can be improved. The following is an explanation of the process of training the initial model for the specified device in the first space: Acquire speech feature data and an initial model corresponding to at least two sample sound signals from a training set, input the speech feature data corresponding to the at least two sample sound signals into the initial model for feature comparison, determine the sample similarity between the at least two sample sound signals, and determine the sample electronic device that picks up the sample sound signal earliest among the at least two sample electronic devices; obtain a similarity label between the speech feature data corresponding to the at least two sample sound signals, train the initial model based on the similarity label, the sample similarity and the sample electronic device, and obtain a speech processing model.

[0067] Among them, at least two sample sound signals may refer to sample sound signals of the speaker picked up by sample electronic devices at different distances from the speaker, and the speech feature data corresponding to the at least two sample sound signals may be obtained by feature extraction of the sample sound signals picked up by each sample electronic device within the sample time window. The similarity label between the speech feature data corresponding to the at least two sample sound signals may include: a sound similarity label and a sample electronic device label, the sound similarity label may be used to reflect whether the at least two sample sound signals are similar, and the sample electronic device label may be used to indicate the electronic device that picks up the sample sound signal earliest among the at least two sample electronic devices, that is, the sample real data. The sample similarity between the at least two sample sound signals and the sample electronic device that picks up the sample sound signal earliest among the at least two sample electronic devices refer to model prediction data. By comparing whether the sample real data and the model prediction data are consistent, it can be determined whether the initial model converges. When the sample real data and the model prediction data are inconsistent, the model parameters in the initial model can be adjusted so that the sample real data and the model prediction data of the initial model are as consistent as possible. When the sample real data and the model prediction data of the initial model are consistent, it is determined that the initial model converges, and the initial model at this time is saved as a speech processing model.

[0068] Optionally, the process of training an initial model based on similarity labels, sample similarity and sample electronic devices to obtain a speech processing model can be as follows: determine a first deviation feature for the initial model based on sound similarity labels and sample similarity, determine a second deviation feature for the initial model based on the sample electronic device and the sample electronic device label, and train the initial model based on the first deviation feature and the second deviation feature to obtain a speech processing model.

[0069] Among them, the process of training the initial model may refer to the process of adjusting the model parameters in the initial model, the first deviation feature and the second deviation feature may refer to the loss function of the initial model, the larger the value of the loss function of the initial model, the lower the prediction accuracy of the initial model, so the model parameters in the initial model can be adjusted to reduce the value of the loss function in the initial model, when the value of the loss function in the initial model is less than the loss threshold, for example, the first deviation feature is less than the first deviation threshold and the second deviation feature is less than the second deviation threshold, or when the number of iterations of the initial model is greater than or equal to the number threshold, or when the training time of the initial model meets the preset time, then the initial model at this time is considered to be converged, and the initial model at this time is saved as the speech processing model. Since the initial model is trained by selecting a large number of speech feature data corresponding to sample sound signals from the training set, the accuracy and stability of the speech processing model can be improved.

[0070] That is to say, in this scenario, the speech processing model can directly compare the speech feature data of at least two electronic devices and output two results. One result is used to indicate whether at least two sample sound signals are similar, that is, the sample similarity result, and the other result is used to indicate the sample electronic device that picks up the sample sound signal earliest among the at least two sample electronic devices. By combining the two results output by the speech processing model, the first electronic device can be determined, and different controls can be performed on the first electronic device and other electronic devices respectively, thereby improving the accuracy of the control of the electronic devices.

[0071] Optionally, when the voice feature data of at least two electronic devices in the first space are compared and it is determined that the voice feature data of at least two electronic devices in the first space are similar, if the voice feature data of at least two electronic devices in the first space all meet the target transmission conditions, then any one of the electronic devices can be selected as the first electronic device. The voice feature data of the electronic device meeting the target transmission conditions may mean that the clarity of the sound signal of the electronic device is greater than the clarity threshold, and the sound signal delay is less than the delay threshold, that is, when the sound signal of the electronic device is transmitted to the electronic device in the second space through the network, the sound signal output by the electronic device in the second space is also clear. For example, if the voice feature data of at least two electronic devices in the first space all meet the target transmission conditions, any one of the electronic devices having a network speed greater than the network speed threshold, or a network remaining flow greater than the flow threshold, or a device performance greater than the performance threshold can be selected from at least two electronic devices as the first electronic device.

[0072] In an embodiment of the present application, for example, if the area of ​​the first space is small, each electronic device in the first space can pick up the voice of the same speaker, and the picked up sound signals are clear and the sound signal delay is low, then a better first electronic device can be selected based on factors such as network speed, remaining traffic, and device performance, so that the first electronic device can subsequently pick up the speaking sound in the first space, thereby improving the sound signal transmission efficiency in the first space.

[0073] S103, controlling the first electronic device to send the picked-up sound signal to other electronic devices, and controlling the second electronic device not to send the picked-up sound signal to other electronic devices.

[0074] Among them, the first electronic device refers to the electronic device used by the speaker in the current time window in the first space, the second electronic device includes electronic devices in the first space other than the first electronic device, and other electronic devices include electronic devices in the first space and electronic devices in the second space. In other words, controlling the first electronic device to send the picked-up sound signal to other electronic devices means controlling the first electronic device to send the picked-up sound signal to both electronic devices in the first space other than the first electronic device and electronic devices in the second space. Controlling the second electronic device not to send the picked-up sound signal to other electronic devices means controlling the second electronic device not to send the picked-up sound signal to electronic devices in the first space other than the first electronic device and electronic devices in the second space.

[0075] It can be understood that the first electronic device can refer to, for example, the main electronic device in the first space, and the second electronic device can refer to, for example, the secondary electronic device in the first space. When there is only one speaker in the first space, the number of main electronic devices in the first space is one, and the number of secondary electronic devices in the first space can be the total number of devices other than the main electronic devices. When there are multiple speakers in the first space, the number of main electronic devices in the first space is multiple, and the number of secondary electronic devices in the first space can be the total number of devices other than the multiple main electronic devices. In the case where there are multiple speakers speaking in the first space within the current time window, the number of first electronic devices can be multiple, and the designated device of the first space can process multiple first electronic devices in the same way.

[0076] Optionally, the electronic devices in the first space can determine the first electronic device in the following manner: perform feature comparison on audio feature information included in voice feature data of at least two electronic devices to determine a first similarity between sound signals picked up by the at least two electronic devices; and when the first similarity is greater than or equal to a first similarity threshold, compare time indication information included in the voice feature data of at least two electronic devices to determine the first electronic device that picks up the sound signal earliest among the at least two electronic devices.

[0077] Among them, performing feature comparison on the audio feature information included in the voice feature data of at least two electronic devices, and determining the first similarity between the sound signals picked up by at least two electronic devices may refer to calculating the similarity distance between the audio feature information included in the voice feature data of at least two electronic devices. The method for calculating the similarity distance can be referred to the introduction in the previous article and will not be repeated here.

[0078] It can be seen that by comparing the audio feature information included in the voice feature data of at least two electronic devices, it can be determined whether the sound signals picked up by the at least two electronic devices are the same or similar. If the sound signals picked up by the at least two electronic devices are not the same or similar, it can be indicated that the sound signals picked up by the two electronic devices are not the sound signals of the same speaker. For example, there are multiple speakers speaking at the same time in the first space, and the electronic devices used by each speaker pick up the speech of each speaker respectively. If the sound signals picked up by the at least two electronic devices are the same or similar, it can be indicated that the sound signals picked up by the at least two electronic devices are the voice of the same speaker. For example, the same sound signals picked up by the two electronic devices can indicate that the distances between the two electronic devices and the speaker are the same. The similar sound signals picked up by the two electronic devices can indicate that the distances between the two electronic devices and the speaker are different. However, they both pick up the speech of the same speaker. Then, the first electronic device that picks up the sound signal earliest can be determined by further judging the time indication information included in the voice feature data of the at least two electronic devices.

[0079] That is to say, when the first similarity is less than the first similarity threshold, there is no need to compare the time indication information included in the voice feature data of at least two electronic devices, which can improve the efficiency of determining the first electronic device. When it is determined that the first similarity is greater than or equal to the first similarity threshold, the time indication information included in the voice feature data of at least two electronic devices is compared to determine the first electronic device that picks up the sound signal earliest among the at least two electronic devices, which can improve the accuracy of the electronic device determination.

[0080] Optionally, the process of performing feature comparison on the audio feature information included in the voice feature data of at least two electronic devices and determining the first similarity between the sound signals picked up by the at least two electronic devices can be implemented by a voice processing model. Specifically, the designated device in the first space can input the audio feature information respectively included in the voice feature data of at least two electronic devices into the voice processing model for feature comparison to determine the first similarity between the sound signals picked up by the at least two electronic devices.

[0081] The speech processing model is obtained by training the initial model based on the similarity label between at least two sample sound signals and the sample similarity between at least two sample sound signals, and the sample similarity is obtained by comparing the sample audio feature information of at least two sample sound signals based on the initial model. The first similarity between the sound signals picked up by at least two electronic devices refers to the model output result, and the similarity label between at least two sample sound signals is the sample true result. By comparing the difference between the model output result and the sample true result, the initial model can be adjusted based on the difference. When the difference between the model output result and the sample true result is less than the difference threshold, or the number of iterations of the initial model is greater than or equal to the number threshold, or the training time of the initial model meets the preset time, it can be determined that the initial model converges, and the initial model at this time is saved as the speech processing model.

[0082] That is to say, when training the initial model, only the initial model can be trained for the ability to calculate the similarity between the sound signals picked up by at least two electronic devices. Then, when using the speech processing model for feature comparison, the first similarity between the sound signals picked up by at least two electronic devices can be output. The ability to determine the first electronic device that picks up the sound signal earliest among at least two electronic devices can be implemented using the initial model or not. When using the initial model to implement the ability to determine the first electronic device that picks up the sound signal earliest among at least two electronic devices, the training process of the initial model can refer to the previous introduction and will not be repeated here.

[0083] In an embodiment of the present application, after the designated device in the first space determines the first electronic device, it controls the first electronic device to send the picked up sound signal to the second electronic device and the electronic device in the second space. After the electronic device in the second space receives the sound signal picked up by the first electronic device, it can output the sound signal picked up by the first electronic device in the second space, thereby realizing the transmission of the sound signal in the first space to the ears of the participants in the second space.

[0084] See also Figure 4 , Figure 4 A schematic diagram of an application scenario for processing echoes in a far-end space provided in an embodiment of the present application, such as Figure 4As shown, there are two electronic devices in the local space, namely, electronic device 41 of participant 1 and electronic device 42 of participant 2. The current state is that participant 2 is speaking, and electronic device 1 of participant 1 has not actively muted the audio input device such as a microphone. Then the voice signal of participant 2 is picked up by electronic device 42, and the sound signal energy is strong and the sound is clear. Electronic device 41 of participant 1 also picks up the voice signal of participant 2, but the sound signal energy is weak and there is a delay. The sound signals picked up by electronic devices 41 and 42 can be transmitted to various electronic devices at this end and electronic devices at the remote end through the cloud. When the sound signals picked up by electronic devices 41 and 42 are transmitted to electronic devices in the remote space through the cloud, the remote space has an echo problem when participant 2 speaks.

[0085] Therefore, by adding audio management modules to the electronic device 41 and the electronic device 42 in the local space respectively, the audio management modules of each electronic device respectively extract the voice feature data of the sound signal picked up by each electronic device, and transmit the voice feature data of each electronic device to the audio management module of the designated device (such as electronic device 42) in the first space based on the communication connection. The audio management module of the designated device in the first space performs feature comparison on the voice feature data of at least two electronic devices, and determines the first electronic device that picks up the sound signal earliest among the at least two electronic devices, that is, the electronic device closest to the speaker. In addition, the audio management module of the designated device in the first space controls the first electronic device to send the picked up sound signal to other electronic devices, and controls the second electronic device not to send the picked up sound signal to other electronic devices. Then, only one electronic device in the first space, namely the first electronic device (for example, the first electronic device is the designated device in the first space, namely the electronic device 42), sends the picked up sound signal to the cloud. Figure 4 As shown in ②, the delayed sound signal picked up by the second electronic device (such as electronic device 41) is not sent to the cloud, that is, the picked up sound signal is not sent to the cloud, then the second space such as the remote space receives the sound signal picked up by the first electronic device and outputs it, so there is no echo signal in the remote space.

[0086] Optionally, after the second electronic device in the first space receives the sound signal picked up by the first electronic device, the second electronic device may determine whether to output the sound signal picked up by the first electronic device in the first space according to its own situation.

[0087] In one case, for example, if the area of ​​the first space is greater than the area threshold, the second electronic device whose distance from the speaker is greater than the distance threshold can be controlled to output the sound signal picked up by the first electronic device; and the second electronic device whose distance from the speaker is less than or equal to the distance threshold can be controlled not to output the sound signal picked up by the first electronic device. Since when the area of ​​the first space is large, it is difficult for the user of the electronic device whose distance from the speaker is greater than the distance threshold to fully hear the speaker's speech, the sound signal picked up by the first electronic device is output by the second electronic device, which can ensure the quality and accuracy of the remote call. If the user of the electronic device whose distance from the speaker is less than or equal to the distance threshold can hear the speaker's speech, there is no need to output the sound signal, and the power consumption of the second electronic device can be reduced while ensuring the quality of the remote call.

[0088] In another case, for example, when the area of ​​the first space is greater than the area threshold, the audio output device of the first electronic device is controlled to output the picked-up sound signal, and the second electronic device is controlled not to output the sound signal picked up by the first electronic device. When the audio output device of the first electronic device outputs the picked-up sound signal, the sound signal can be amplified, so that when the audio output device of the first electronic device outputs the picked-up sound signal, each participant in the first space can clearly hear the sound signal, thereby ensuring the accuracy of the remote call.

[0089] It can be understood that if there are multiple electronic devices sharing the second space in the second space, any electronic device can be selected from the second space as the designated device of the second space in the same manner as that of selecting a designated device from the first space, so that the designated device of the second space can receive the voice feature data sent by each electronic device in the second space to determine the first electronic device in the second space, that is, the device used by the speaker in the second space, thereby implementing different control operations for different electronic devices in the second space. The specific control method can refer to the control operation for the electronic device in the first space, and no further description will be given here.

[0090] In the embodiment of the present application, since the first electronic device, that is, the device used by the speaker in the current time window, is determined from the first space, the electronic device is closest to the speaker. Therefore, the sound signal picked up by the first electronic device in the current time window is clearer than that of the second electronic device in the first space, and the delay of the sound signal is the lowest. By controlling the first electronic device to send the picked up sound signal to other electronic devices, and controlling the second electronic device not to send the picked up sound signal to other electronic devices, the sound signal received by the electronic device receiving the sound signal can be clear and has the lowest delay, that is, the sound effect is the best, thereby solving the echo problem caused by the electronic device receiving the sound signal receiving multiple sound signals with time differences, thereby improving the call quality in remote calls.

[0091] Optionally, when the designated device in the first space is the first electronic device, controlling the first electronic device to send the picked-up sound signal to other electronic devices means that the first electronic device sends the picked-up sound signal to other electronic devices. When the designated device in the first space is not the first electronic device, controlling the first electronic device to send the picked-up sound signal to other electronic devices means that the designated device in the first space controls the first electronic device to send the picked-up sound signal to other electronic devices.

[0092] Optionally, the electronic device selected in the first space controls the first electronic device to send the picked-up sound signal to other electronic devices in the following manner: a first control instruction is sent to the first electronic device; the first control instruction is used to control the first electronic device to send the picked-up sound signal to other electronic devices.

[0093] By sending the first control instruction to the first electronic device, the first electronic device can control the first electronic device to send the picked-up sound signal to other electronic devices based on the first control instruction.

[0094] Optionally, the electronic device selected in the first space controls the second electronic device not to send the picked-up sound signal to other electronic devices in the following manner: sending a second control instruction to the second electronic device; the second control instruction is used to control the second electronic device not to send the picked-up sound signal to other electronic devices.

[0095] By sending the second control instruction to the second electronic device, the second electronic device can control the second electronic device not to send the picked-up sound signal to other electronic devices based on the second control instruction. By sending different control instructions to the first electronic device and the second electronic device respectively, the first electronic device and the second electronic device can know whether to send the picked-up sound signal to other electronic devices, thereby avoiding the first electronic device and the second electronic device both sending the picked-up sound signal to other electronic devices to cause echo.

[0096] Optionally, when sending the first control instruction to the first electronic device, the initial working state of the first electronic device in the current time window may be determined first, thereby determining the control instruction to be sent to the first electronic device in combination with the initial working state of the first electronic device in the current time window. For example, the electronic device selected in the first space may obtain the initial working state of the first electronic device in the current time window; the initial working state is used to indicate whether to send the picked-up sound signal to other electronic devices; when the initial working state of the first electronic device indicates not to send the picked-up sound signal to other electronic devices, the first control instruction is sent to the first electronic device.

[0097] Among them, the initial working state of each electronic device may include the initial working state of the audio input device of each electronic device, such as a microphone, and the initial working state of the audio output device, such as a speaker. The initial working state of each electronic device can be obtained by each electronic device reading the flag bit of its own audio input device, such as a microphone control, and the flag bit of the audio output device, such as a speaker control. By reading the flag bit of its own microphone control and the flag bit of the speaker control, the initial working state of its own microphone and speaker can be determined, so that its own initial working state can be transmitted to the designated device in the first space through the communication connection.

[0098] By determining the control instruction for the first electronic device in combination with the initial working state of the first electronic device in the current time window, the control accuracy for the first electronic device can be improved.

[0099] Optionally, when sending the second control instruction to the second electronic device, the initial working state of the second electronic device in the current time window can be determined first, so as to determine the control instruction to be sent to the second electronic device in combination with the initial working state of the second electronic device in the current time window. For example, the electronic device selected in the first space can obtain the initial working state of the second electronic device in the current time window; the initial working state is used to indicate whether to send the picked-up sound signal to other electronic devices; when the initial working state of the second electronic device indicates to send the picked-up sound signal to other electronic devices, the second control instruction is sent to the second electronic device.

[0100] By determining the control instruction for the second electronic device in combination with the initial working state of the second electronic device in the current time window, the control accuracy for the second electronic device can be improved.

[0101] In the embodiment of the present application, by determining the first electronic device in the first space, the first electronic device can be controlled to send the picked-up sound signal to other electronic devices, so that the sound signal output by the electronic device in the second space is clearer. By controlling the second electronic device not to send the picked-up sound signal to other electronic devices, it is possible to avoid sending multiple sound signals with time differences to the electronic device in the second space, causing the problem of echo signals in the second space, such as the remote space, to exist.

[0102] Next, combine Figure 5 For a detailed description of the specific process of the device management method provided in the embodiment of the present application, please refer to Figure 5 , Figure 5 A schematic diagram of a device management method provided in an embodiment of the present application Figure 2 , Figure 5 It is mainly to solve the echo signal problem of the local space during long-distance calls, such as Figure 5 As shown, the method may include but is not limited to the following steps: S201: Acquire voice feature data of at least two electronic devices in a first space.

[0103] S202: Determine a first electronic device based on voice feature data of at least two electronic devices.

[0104] Among them, the specific implementation method of step S201~step S202 can refer to the implementation method of the aforementioned step S101~step S102, which will not be repeated here.

[0105] Optionally, after determining the first electronic device in the first space, the designated device in the first space may control the audio output device of the second electronic device not to output the sound signal of the first space picked up in the current time window.

[0106] In the embodiment of the present application, since the designated device of the first space determines the first electronic device used by the speaker in the first space, and the first electronic device sends a picked-up sound signal to other electronic devices, when the second electronic device in the first space receives the picked-up sound signal sent by the first electronic device, it can output the picked-up sound signal sent by the first electronic device in the first space. However, since the speaker speaks in the first space, the users of all electronic devices in the first space, that is, all participants in the first space, can hear the speaker's voice. Therefore, there will be the speaker's speech sound propagated through the air in the first space, and there will also be the sound signal picked up by the first electronic device received by the second electronic device through the network. Since the speed of sound signal propagation through the air is greater than the speed of transmission through the network, an echo signal with a time difference will be generated in the first space, and all participants in the first space will hear the echo signal with a time difference, affecting the quality of remote calls.

[0107] Therefore, the designated device in the first space controls the audio output device of the second electronic device not to output the sound signal of the first space picked up within the current time window. Then, the second electronic device in the first space does not output the sound signal of the first space picked up within the current time window. In the first space, only the sound signal of the user of the first electronic device (i.e., the speaker) transmitted through the air is transmitted. Therefore, no echo signal is generated in the first space, thereby improving the clarity of remote calls in the first space.

[0108] Optionally, the audio output device of the second electronic device may be muted, so that the audio output device of the second electronic device does not output the sound signal picked up in the current time window.

[0109] Optionally, the designated device in the first space may control the audio output device of the second electronic device to output the sound signal of the second space picked up by the electronic device in the second space within the current time window.

[0110] That is to say, in the current time window, if there are two speakers speaking simultaneously in the first space and the second space, each electronic device in the first space can output the sound signal of the speaker picked up in the second space, and each electronic device in the second space can output the sound signal of the speaker picked up in the first space, thereby avoiding omission of sound signals in the first space and the second space and improving the accuracy of remote calls.

[0111] Optionally, when controlling the audio output device of the second electronic device not to output the sound signal of the first space picked up in the current time window, the audio output device of the second electronic device can be directly controlled to be muted, so that the audio output device of the second electronic device does not output all the sound signals picked up in the current time window. However, since there may be a speaker in the second space speaking in the current time window, it is also possible to further determine whether only the speaker in the local space is speaking in the current time window, so as to avoid missing the sound signal in the far-end space by directly controlling the audio output device of the second electronic device not to output the sound signal picked up in the current time window when there is also a speaker speaking in the far-end space.

[0112] S203: Acquire audio feature information of the sound signal received by the second electronic device.

[0113] S204, performing feature comparison on the audio feature information of the sound signal received by the second electronic device and the audio feature information included in the voice feature data of the first electronic device, and determining a second similarity between the sound signal received by the second electronic device and the sound signal picked up by the first electronic device.

[0114] S205, when the second similarity is greater than or equal to the second similarity threshold, determine that the sound signal received by the second electronic device is a sound signal of the first space picked up in the current time window, and control the audio output device of the second electronic device not to output the received sound signal.

[0115] The audio feature information of the sound signal received by the second electronic device is obtained by extracting the features of the sound signal received by the second electronic device. The sound signal received by the second electronic device may include the sound signal of the first space picked up by the first electronic device in the current time window sent by the first electronic device through the network, and the sound signal of the second space picked up by the electronic device in the second space in the current time window sent by the network. The audio feature information is obtained by extracting the features of the two different types of sound signals respectively. By comparing the audio feature information of the sound signal received by the second electronic device with the audio feature information included in the voice feature data of the first electronic device, it can be determined whether the sound content of the sound signal received by the second electronic device is the same as that of the sound signal picked up by the first electronic device in the current time window, that is, whether it is the speech content of the same speaker. In the case where the second similarity between the sound signal received by the second electronic device and the sound signal picked up by the first electronic device is greater than or equal to the second similarity threshold, it means that the sound signal received by the second electronic device is sent by the sound signal picked up by the first electronic device through the network, and there is no speaker in the second space in the current time window, and the audio output device of the second electronic device can be controlled not to output the received sound signal, for example, the audio output device of the second electronic device can be directly muted.

[0116] Optionally, feature comparison may be performed between the audio feature information of the sound signal received by the second electronic device and the audio feature information included in the voice feature data of the second electronic device to determine a second similarity between the sound signal received by the second electronic device and the sound signal picked up by the second electronic device.

[0117] Since it is determined through the above steps that the sound signals of the first electronic device and the second electronic device are similar, that is, they are sound signals of the same speaker picked up by the two electronic devices respectively, the result of the feature comparison is the same regardless of whether the voice feature data of the first electronic device or the audio feature information included in the voice feature data of the second electronic device is used for feature comparison with the audio feature information of the sound signal received by the second electronic device.

[0118] Optionally, since the first electronic device and the second electronic device are located in the first space, when the second electronic device receives a sound signal, the first electronic device will also receive the same sound signal. When it is determined that the sound signal received by the second electronic device is a sound signal of the first space picked up within the current time window, the audio output device of the first electronic device can also be controlled not to output the received sound signal, thereby avoiding the generation of an echo signal in the first space.

[0119] That is to say, when each electronic device in the first space receives a sound signal sent by an electronic device in the first space, the audio output device of each electronic device in the first space does not output the sound signal, so that only the sound signal of the speaker transmitted through the air is transmitted in the first space, so as to avoid the generation of an echo signal in the local space.

[0120] S206: When the second similarity is less than the second similarity threshold, determine that the sound signal received by the second electronic device is a sound signal of the second space picked up in the current time window, and control the audio output device of the second electronic device to output the received sound signal.

[0121] Since the second similarity between the sound signal received by the second electronic device and the sound signal picked up by the first electronic device is less than the second similarity threshold, it means that the sound signal received by the second electronic device is not the sound signal picked up by the first electronic device and sent through the network, but the electronic device in the second space picks up the sound signal of the speaker in the second space within the current time window and sends it through the network. Therefore, the audio output device of the second electronic device is controlled to output the received sound signal, thereby avoiding omission of call content in the second space and improving the accuracy of remote calls.

[0122] See also Figure 6 , Figure 6 A schematic diagram of an application scenario for processing echoes in a local space provided in an embodiment of the present application, such as Figure 6 As shown, there are two electronic devices in the local space, and the current state is that participant 2 is speaking, and the electronic device 1 of participant 1 has not actively muted the audio output device such as a speaker. Then the voice signal of participant 2 is picked up by the electronic device 62 and transmitted through the cloud, and the audio output device of the electronic device 61 of participant 1, such as a speaker, is played, and participant 2 hears it. At the same time, participant 1 can also hear the voice signal of participant 2 transmitted through the air. Since there is a delay between the voice signal transmitted through the cloud and the voice signal transmitted through the air, all participants at the local end, such as participant 1 and participant 2, can clearly hear the echo signal.

[0123] Therefore, by adding audio management modules to each electronic device in the local space, such as electronic device 61 and electronic device 62, the audio management module of each electronic device 61 can obtain the sound signal transmitted through the network, and extract the features of the sound signal to obtain voice feature data, and transmit the respective voice feature data to the audio management module of the designated device 62 (such as electronic device 62) in the first space based on the communication connection. The audio management module of the designated device in the first space performs feature comparison on the voice feature data of at least two electronic devices, and determines that the sound signal received by the second electronic device is the sound signal of the first space picked up within the current time window, that is, the sound signal received by all second electronic devices is the sound signal transmitted by the first electronic device through the network, and then mutes the audio output devices of all second electronic devices, such as controlling the audio output device of the second electronic device (such as electronic device 61) not to output the received sound signal. Then, Figure 6 As shown in ③, there are only sound signals transmitted through the air in the local space, so there is no echo signal in the local space.

[0124] Optionally, if it is detected in the first space that any electronic device has picked up a sound signal within any time window, and it is determined that the electronic device in the second space has not picked up a sound signal within the any time window, it means that the sound signal received by each electronic device in the first space is picked up by the any electronic device in the first space, then the audio output devices of all electronic devices in the first space are controlled not to output the received sound signal, thereby reducing the echo signal generated by this end. In other words, when a participant in the first space is speaking, if it is determined that no participant in the second space is speaking at this time, all speakers in the first space can be muted, thereby reducing the echo generated in the first space.

[0125] Optionally, since the first electronic device and the second electronic device are located in the first space, when the second electronic device receives a sound signal, the first electronic device will also receive the same sound signal. When it is determined that the sound signal received by the second electronic device is a sound signal of the second space picked up within the current time window, the audio output device of the first electronic device can also be controlled to output the received sound signal to avoid the problem of missing speech sounds in the second space such as the far-end space in a remote call.

[0126] That is, when each electronic device in the first space receives a sound signal sent by an electronic device in the second space, the audio output device of each electronic device in the first space outputs the sound signal to avoid missing the sound signal in the far-end space.

[0127] Optionally, if a sound signal is received within any time window, the processing method for the sound signal can be determined in combination with the electronic device to which the received sound signal belongs, and any time window can include the current time window or other time windows. For example, a designated device in the first space receives a sound signal sent by a third electronic device and picked up by a third electronic device; when the third electronic device is an electronic device in the second space, the audio output device of the second electronic device is controlled to output the sound signal picked up by the third electronic device.

[0128] For example, it can be determined whether the third electronic device belongs to the first space or the second space by obtaining the identification of the third electronic device. The identification of the third electronic device may include, but is not limited to, the participant identity information of the third electronic device, the Internet Protocol Address (IP address) of the third electronic device, the location of the organization to which the third electronic device belongs, the factory number of the third electronic device, and so on. Since the sound signal sent by the remote space and picked up by the third electronic device in the remote space is received, the audio output devices of all the second electronic devices in the local space can be unmuted, so that the audio output devices of all the second electronic devices in the local space output the sound signal picked up by the third electronic device, thereby reducing the omission of the sound signal in the remote space and ensuring the accuracy of the remote call.

[0129] Optionally, when the third electronic device is an electronic device in the first space, the audio output device of the second electronic device is controlled not to output the sound signal picked up by the third electronic device. Since the received sound signal is a sound signal picked up by the local space, in order to reduce the echo signal generated in the local space, the audio output device of the second electronic device is controlled not to output the sound signal picked up by the third electronic device.

[0130] Optionally, when it is determined that the second similarity between the sound signal sent by the third electronic device and picked up by the third electronic device and the sound signal picked up by the first electronic device is small, it can be further determined whether the third electronic device is an electronic device in the second space to determine the processing method for the audio output device of the third electronic device, thereby determining whether to output the sound signal sent by the third electronic device in the first space.

[0131] Specifically, a feature comparison is performed on the audio feature information of the sound signal sent by the third electronic device and the audio feature information included in the voice feature data of the first electronic device to determine the second similarity between the sound signal sent by the third electronic device and the sound signal picked up by the first electronic device; when the second similarity is less than a second similarity threshold and the third electronic device is an electronic device in the second space, the audio output device of the second electronic device is controlled to output the sound signal sent by the third electronic device.

[0132] Among them, the sound signal sent by the third electronic device refers to the sound signal sent by the third electronic device and picked up by the third electronic device. When receiving a sound signal sent by any electronic device, by combining the sound signal sent by the electronic device with the sound signal picked up by the first electronic device in the local space, and further combining the space to which the electronic device belongs to determine whether to output the sound signal sent by the electronic device, the accuracy of the sound signal output can be improved. For example, when receiving a sound signal sent by a third electronic device, by further combining the space to which the third electronic device belongs to determine whether the sound signal sent by the third electronic device is a sound signal sent by the far-end space or a sound signal sent by the local space, the accuracy of the sound signal judgment can be improved to avoid the situation where the sound signals picked up by each electronic device are inaccurate when there are multiple sound signals on the local end, resulting in inaccurate voice similarity comparison, thereby causing the far-end signal in the remote call to be missed, and the accuracy of the remote call can be improved.

[0133] Optionally, when the second similarity is less than a second similarity threshold and the second electronic device is an electronic device in the first space, the audio output device of the second electronic device is controlled not to output the sound signal picked up by the second electronic device.

[0134] In the embodiment of the present application, when the second similarity is less than the second similarity threshold and the second electronic device is an electronic device in the first space, it means that the sound signal received by the second electronic device is a sound signal sent by the local space. In this case, the audio output device of the second electronic device is controlled not to output the sound signal picked up by the second electronic device to avoid generating an echo signal in the local space.

[0135] It can be seen that, through the embodiment of the present application, by controlling the audio output device of the second electronic device not to output the sound signal of the first space picked up in the current time window, no echo signal will be generated in the first space, thereby solving the echo problem in the first space during the remote call and improving the clarity of the remote call. In addition, by further analyzing the second similarity between the sound signal received by the second electronic device and the sound signal picked up by the first electronic device, the accuracy of determining whether the sound signal received by the second electronic device is the sound signal of the first space picked up in the current time window or the sound signal received by the second electronic device is the sound signal of the second space picked up in the current time window can be improved, thereby improving the accuracy of controlling the audio output device of the second electronic device.

[0136] It is understandable that Figure 2 In the embodiment of the first embodiment, the solution for the echo signal in the first space and the solution for the echo signal in the first space Figure 5In the embodiment of the present invention, the solution for the echo signal in the second space can be executed synchronously, and the execution processes of the two solutions do not affect each other. The solution for the echo signal in the first space can also be executed first, and then the solution for the echo signal in the second space can be executed, or the solution for the echo signal in the second space can be executed first, and then the solution for the echo signal in the first space can be executed. The embodiment of the present application is not limited to this.

[0137] Above Figure 2 , Figure 5 The embodiment of the first space specifies the process of managing the device of the first space. Figure 7 , for the above Figure 2 , Figure 5 In the embodiment of the present invention, the flowchart of the interaction between the modules in the process of device management of the designated device in the first space is used as an example. Figure 7 , Figure 7 The present invention provides a flowchart of the interaction between modules of an electronic device. In the process of device management of the electronic device, the interaction process of the modules in the electronic device is as follows: S301: Determine a designated device from a first space.

[0138] The designated device refers to the designated device of the first space. For example, the designated device of the first space can be determined from the first space, or any electronic device in the first space can determine the designated device from the first space. After the designated device is determined, a communication connection is established between the audio management module of the designated device and the audio management modules of multiple electronic devices in the first space. Then, the audio management module of the designated device can receive the voice feature data sent by the audio management modules of other electronic devices for analysis, and determine the first electronic device used by the speaker.

[0139] Optionally, Figure 7 The multiple electronic devices in the first space may include, for example, a designated device in the first space, a first electronic device, and a second electronic device. After determining the designated device, the audio management module of the designated device may establish a connection with the audio management modules of the electronic devices in the first space.

[0140] S302: Establish a communication connection between the audio management module of the designated electronic device and the audio management module of the first electronic device.

[0141] S303: Establish a communication connection between the audio management module of the designated electronic device and the audio management module of the second electronic device.

[0142] S304: The audio input device of the designated device picks up the sound signal in the first space and transmits it to the audio management module of the designated device.

[0143] S305: The audio management module of the designated device extracts features from the sound signal to obtain voice feature data.

[0144] The audio input device of each electronic device may pick up the sound signal in the first space and transmit it to its own audio management module, which then extracts features of the sound signal picked up by itself to obtain voice feature data.

[0145] S306: The audio input device of the first electronic device picks up the sound signal in the first space and transmits it to the audio management module of the first electronic device.

[0146] S307: The audio management module of the first electronic device extracts features from the sound signal to obtain voice feature data, and transmits the data to the audio management module of the designated device.

[0147] S308: The audio input device of the second electronic device picks up the sound signal in the first space and transmits it to the audio management module of the second electronic device.

[0148] S309: The audio management module of the second electronic device extracts features from the sound signal to obtain voice feature data, and transmits the data to the audio management module of the designated device.

[0149] Among them, each electronic device can transmit voice feature data to the audio management module of the designated device based on the communication connection.

[0150] S310: An audio management module of a designated device determines a first electronic device based on voice feature data of a plurality of electronic devices.

[0151] S311: The audio management module of the designated device sends a first control instruction to the audio management module of the first electronic device.

[0152] Among them, the audio management module of the designated device can send a first control instruction to the audio management module of the first electronic device, and the first control instruction is used to control the first electronic device to send the picked up sound signal to other electronic devices, for example, to send the picked up sound signal to the second electronic device and the electronic device in the second space.

[0153] S312: The audio management module of the first electronic device sends the picked-up sound signal to the audio management module of the second electronic device.

[0154] S313: The audio output device of the second electronic device outputs the received sound signal.

[0155] The second electronic device may output the received sound signal through an audio output device such as a speaker.

[0156] S314: The audio management module of the first electronic device sends the picked-up sound signal to the audio output device of the electronic device in the second space.

[0157] Among them, the audio management module of the first electronic device can send the picked-up sound signal to the electronic device in the second space. If the number of electronic devices in the second space is one, the audio management module may not be added to the electronic device in the second space, and the audio management module of the first electronic device can send the picked-up sound signal to the electronic device in the second space or the audio output device of the electronic device in the second space. If the number of electronic devices in the second space is multiple, that is, there are multiple electronic devices sharing the space in the second space, an audio management module can be added to each electronic device in the second space, and the audio management module of the first electronic device can send the picked-up sound signal to the audio management modules on each electronic device in the second space, and the audio management modules on each electronic device in the second space can output the received sound signal through the audio output device.

[0158] S315: The audio output device of the electronic device in the second space outputs the received sound signal.

[0159] For example, the electronic device in the second space may output the received sound signal through an audio output device such as a speaker.

[0160] It can be understood that if the designated device of the first space is not the first electronic device, when the audio management module of the first electronic device sends the picked up sound signal to the second electronic device, it also includes sending the picked up sound signal to the audio output device of the designated device, and the audio output device of the designated device outputs the received sound signal.

[0161] It can be understood that if the designated device in the first space is the first electronic device, the audio output device of the designated device may output or not output the sound signal picked up by itself.

[0162] S316: The audio management module of the designated device sends a second control instruction to the audio management module of the second electronic device.

[0163] S317: The audio management module of the second electronic device does not send the picked-up sound signal to other electronic devices.

[0164] Among them, the audio management module of the designated device can send a second control instruction to the audio management module of the second electronic device, and the second control instruction is used to control the audio management module of the second electronic device not to send the picked up sound signal to other electronic devices, that is, the second electronic device does not send the picked up sound signal to the first electronic device, and the second electronic device does not send the picked up sound signal to the electronic device in the second space.

[0165] Optionally, the method of controlling the second electronic device not to send the picked-up sound signal to other electronic devices may include the following two methods: The first way: the OS system (Operating System) of the second electronic device receives the second control instruction sent by the designated device based on the communication connection, and the OS system of the second electronic device turns off the audio input device such as a microphone.

[0166] The second method: the audio management module of the second electronic device intercepts the path for the second electronic device to transmit sound signals to the network, so that the second electronic device does not transmit sound signals to the network. It is understandable that the path for the second electronic device to transmit sound signals to the network is always open during the entire remote call process. When controlling the second electronic device not to transmit sound signals to the network, it is not necessary to close the path, but only to intercept the sound signal.

[0167] It can be understood that in the embodiment of the present application, when the second electronic device is controlled not to send the picked-up sound signal to other electronic devices, the second electronic device can still pick up the sound signal of the first space, and can perform feature extraction on the sound signal in each time window to obtain voice feature data and transmit it to the designated device of the first space based on the communication connection. The designated device of the first space determines whether the speaker has changed, that is, whether the first electronic device has changed, so that even if the first electronic device used by the changed speaker is controlled to send the picked-up sound signal to other electronic devices, and the second electronic device is controlled not to send the picked-up sound signal to other electronic devices, the clarity of the remote call can be improved.

[0168] S318: The audio management module of the designated device sends a third control instruction to the audio management module of the second electronic device.

[0169] S319: The audio output device of the second electronic device does not output the sound signal of the first space picked up in the current time window.

[0170] The third control instruction is used to control the audio output device of the second electronic device not to output the sound signal of the first space picked up in the current time window. For example, the audio management module of the designated device can control the audio output device of the second electronic device not to output the sound signal of the first space picked up in the current time window.

[0171] It can be understood that controlling the audio output device of the second electronic device not to output the sound signal of the first space picked up within the current time window may refer to muting the audio output device of the second electronic device. After muting the audio output device of the second electronic device, the audio output device of the second electronic device can still receive sound signals sent by the first electronic device and the electronic device in the second space transmitted through the network, but does not output the received sound signals, that is, does not play the received sound signals.

[0172] Optionally, the audio input device of the electronic device in the second space picks up the sound signal in the second space and transmits it to the audio management modules of multiple electronic devices in the first space. The audio management modules of the multiple electronic devices in the first space control their own audio output devices to output the sound signal in the second space.

[0173] Optionally, Figure 7 The implementation methods not mentioned in steps S301 to S319 in the corresponding embodiments can refer to the specific implementations of the aforementioned steps S101 to S103, and the aforementioned steps S201 to S206, which will not be repeated here.

[0174] Through the device management method in the embodiment of the present application, echo path elimination and echo suppression of remote calls in a shared space can be achieved. By establishing a communication connection between various electronic devices in the same space, the transmission of voice feature data can be achieved, thereby controlling the recognition of the first electronic device, i.e., the main electronic device, and the second electronic device, i.e., the secondary electronic device, and controlling the main electronic device to send the picked-up sound signal to other devices, and controlling the secondary electronic device not to send the picked-up sound signal to other devices, thereby reducing the far-end echo signal in the remote call. And when the user of the main electronic device is speaking, the speaker of the secondary electronic device is controlled not to output the first seen sound signal picked up by the main electronic device, thereby reducing the local echo signal in the remote call.

[0175] The hardware structure of the electronic device 100 is introduced below: See also Figure 8 , Figure 8 A schematic diagram of the hardware structure of the electronic device 100 provided in an embodiment of the present application.

[0176] The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0177] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0178] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0179] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0180] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0181] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0182] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0183] The charging management module 140 is used to receive charging input from a charger, where the charger can be a wireless charger or a wired charger.

[0184] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. In some other embodiments, the power management module 141 may also be disposed in the processor 110.

[0185] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0186] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0187] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0188] The modulation and demodulation processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor.

[0189] The wireless communication module 160 can provide wireless communication solutions including Wi-Fi network, Bluetooth, BLE broadcast, global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0190] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0191] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0192] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1. The display screen 194 may include an OLED screen.

[0193] Optionally, the display screen 194 may further include: an OLED glass layer, an OLED light-emitting unit, a fingerprint recognition sensor, a microlens array, etc. The display screen 194 supports optical under-screen fingerprint recognition.

[0194] The electronic device 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor. The ISP is used to process the data fed back by the camera 193. The camera 193 is used to capture static images or videos. The camera 193 may include a front camera and a rear camera, the front camera is located in the display area of ​​the screen, and the rear camera is located in the back area of ​​the screen. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. The video codec is used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs.

[0195] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. NPU can realize applications such as intelligent cognition of electronic device 100, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0196] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing compressed driver files and other files in the external memory card.

[0197] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as an audio playback function), etc. The data storage area may store data (such as a video stream) created during the use of the electronic device 100, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as a flash memory device, etc. Optionally, the code of the audio management module can be read into the internal memory 121, and the processor 110 executes operations such as determining the first electronic device in the first space, sending a first control instruction to the first electronic device, and sending a second control instruction to the second electronic device by running the instructions stored in the internal memory 121.

[0198] The electronic device 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D, and the application processor, etc. For example, the speaker 170A outputs the sound signal of the speaker in the first space, outputs the sound signal transmitted in the far-end space, and the microphone 170C collects the sound signal of the speaker, etc.

[0199] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0200] Speaker 170A, also known as "speaker", is used to convert audio electrical signals into sound signals. Receiver 170B, also known as "earpiece", is used to convert audio electrical signals into sound signals. Microphone 170C, also known as "microphone" and "microphone", is used to convert sound signals into electrical signals. Headphone interface 170D is used to connect wired headphones. Pressure sensor 180A is used to sense pressure signals and can convert pressure signals into electrical signals. In some embodiments, pressure sensor 180A can be set on display screen 194. Gyroscope sensor 180B can be used to determine the motion posture of electronic device 100. Air pressure sensor 180C is used to measure air pressure. Magnetic sensor 180D includes Hall sensor. Acceleration sensor 180E can detect the magnitude of acceleration of electronic device 100 in various directions (generally three axes). Distance sensor 180F is used to measure distance. Proximity light sensor 180G can include, for example, light emitting diode (LED) and light detector. Ambient light sensor 180L is used to sense ambient light brightness. Fingerprint sensor 180H is used to collect fingerprints. The temperature sensor 180J is used to detect the temperature. The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The bone conduction sensor 180M can obtain vibration signals. The buttons 190 include a power button, a volume button, etc. The motor 191 can generate vibration prompts. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power changes, and can also be used to indicate messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect a SIM card.

[0201] The following is an introduction to the software structure of electronic equipment: Fig. 9Schematic diagram of the software structure of the electronic device 100 of the embodiment of the present application. The software structure adopts a layered architecture, which divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In the embodiment of the present application, the electronic device is an electronic device installed with Windows (operating system) as an example, and the electronic device may include: application layer (application, APP), framework layer, kernel layer (Kernal), hardware layer (Hardware).

[0202] The application layer can include a series of application packages. Fig. 9 As shown, the application package may include applications such as conference, live broadcast, camera, call, video, etc.

[0203] The framework layer refers to a layer used for audio management, such as a layer used for communication and interaction between audio management modules, for example, it can control whether the sound signal picked up by the electronic device is transmitted or not transmitted to the network, and control the transmission of voice feature data between different electronic devices, etc. For example, the framework layer may include an audio management module and a local communication module, etc.

[0204] Among them, the audio management module is used to extract features of sound signals to obtain voice feature data, analyze the voice feature data of multiple electronic devices to determine the first electronic device, control the first electronic device to send the picked up sound signals to other electronic devices, and control the second electronic device not to send the picked up sound signals to other electronic devices, etc.

[0205] The local communication module is used to realize data transmission between multiple electronic devices in the same space, such as voice feature data transmission, initial working state transmission of speakers and microphones, etc. The local communication module may include, for example, a WiFi module, a Bluetooth module, etc. Both the WiFi module and the Bluetooth module are used to transmit voice feature data, initial working states of speakers and microphones.

[0206] The kernel layer is responsible for managing the system's hardware resources and providing necessary services to applications. The kernel layer includes drivers for various hardware devices, which are responsible for controlling and operating the hardware devices. The kernel layer may include audio input device drivers and audio output device drivers.

[0207] The audio input device driver is used to drive the audio input device to pick up sound signals and upload them to the network or not. For example, the audio input device driver may refer to a microphone driver.

[0208] The audio output device driver is used to drive the audio output device to output or not output the received sound signal. For example, the audio output device driver may refer to a speaker driver.

[0209] The hardware layer includes a plurality of hardware, such as an audio input device and an audio output device, wherein the audio input device may be, for example, a microphone, and the audio output device may be, for example, a speaker.

[0210] Among them, audio input devices such as microphones are used to collect sound signals in the space where the electronic device is located, and audio output devices such as speakers are used to play the received sound signals, or the speakers can also amplify the sound signals picked up by the microphone of the electronic device and output them in the space.

[0211] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive solid statedisk), etc.

[0212] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0213] In short, the above is only an embodiment of the technical solution of this application, and is not intended to limit the protection scope of this application. Any modification, equivalent replacement, improvement, etc. made according to the disclosure of this application shall be included in the protection scope of this application.

Claims

1. A device management method, characterized in that: The method comprises: Acquire voice feature data of at least two electronic devices in the first space; the voice feature data of each electronic device is obtained by extracting features of a sound signal picked up by each electronic device in a current time window; Determine a first electronic device based on the voice feature data of the at least two electronic devices, and control the first electronic device to send the picked-up sound signal to other electronic devices, and control the second electronic device not to send the picked-up sound signal to the other electronic devices; the second electronic device includes electronic devices other than the first electronic device in the first space, and the other electronic devices include electronic devices in the first space and electronic devices in the second space; The audio output device of the second electronic device is controlled not to output the sound signal of the first space picked up in the current time window.

2. The method according to claim 1, characterized in that The determining the first electronic device based on the voice feature data of the at least two electronic devices comprises: Performing feature comparison on the audio feature information included in the voice feature data of the at least two electronic devices to determine a first similarity between the sound signals picked up by the at least two electronic devices; When the first similarity is greater than or equal to a first similarity threshold, time indication information included in the voice feature data of the at least two electronic devices is compared to determine a first electronic device among the at least two electronic devices that picks up the sound signal earliest.

3. The method according to claim 2, characterized in that The step of performing feature comparison on the audio feature information included in the voice feature data of the at least two electronic devices to determine a first similarity between the sound signals picked up by the at least two electronic devices includes: Inputting the audio feature information respectively included in the voice feature data of the at least two electronic devices into the voice processing model for feature comparison, and determining a first similarity between the sound signals picked up by the at least two electronic devices; Among them, the speech processing model is obtained by training an initial model based on the similarity label between at least two sample sound signals and the sample similarity between the at least two sample sound signals, and the sample similarity is obtained by feature comparison of sample audio feature information of the at least two sample sound signals based on the initial model.

4. The method according to claim 1, characterized in that: The controlling the audio output device of the second electronic device not to output the sound signal of the first space picked up in the current time window includes: Acquiring audio feature information of the sound signal received by the second electronic device; Performing feature comparison on the audio feature information of the sound signal received by the second electronic device and the audio feature information included in the voice feature data of the first electronic device, and determining a second similarity between the sound signal received by the second electronic device and the sound signal picked up by the first electronic device; When the second similarity is greater than or equal to a second similarity threshold, it is determined that the sound signal received by the second electronic device is the sound signal of the first space picked up within the current time window, and the audio output device of the second electronic device is controlled not to output the received sound signal.

5. The method according to claim 4, characterized in that The method further comprises: When the second similarity is less than the second similarity threshold, it is determined that the sound signal received by the second electronic device is a sound signal of the second space picked up within the current time window, and the audio output device of the second electronic device is controlled to output the received sound signal.

6. The method according to claim 1, characterized in that The controlling the first electronic device to send the picked-up sound signal to other electronic devices, and controlling the second electronic device not to send the picked-up sound signal to the other electronic devices, comprises: Sending a first control instruction to the first electronic device; the first control instruction is used to control the first electronic device to send the picked-up sound signal to the other electronic devices; A second control instruction is sent to the second electronic device; the second control instruction is used to control the second electronic device not to send the picked-up sound signal to the other electronic devices.

7. The method according to claim 6, characterized in that The sending a first control instruction to the first electronic device includes: Acquire an initial working state of the first electronic device within the current time window; the initial working state is used to indicate whether to send the picked-up sound signal to the other electronic devices; When the initial working state of the first electronic device indicates not to send the picked-up sound signal to the other electronic devices, the first control instruction is sent to the first electronic device.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: receiving a sound signal sent by a third electronic device and picked up by the third electronic device; In a case where the third electronic device is an electronic device in the second space, the audio output device of the second electronic device is controlled to output the sound signal picked up by the third electronic device.

9. The method according to any one of claims 1 to 7, characterized in that: The acquiring of voice feature data of at least two electronic devices in the first space includes: Establishing a communication connection with at least two electronic devices in the first space; The voice feature data transmitted respectively by the at least two electronic devices based on the communication connection are received.

10. An electronic device, characterized in that: The electronic device comprises: one or more processors, a memory and a touch screen; the memory is used to store program code; the processor is used to run the program code, so that the electronic device implements the method according to any one of claims 1-9.

11. A chip system, applied to electronic equipment, characterized in that: The chip system includes at least one processor and an interface, wherein the interface is used to receive instructions and transmit them to the at least one processor; the at least one processor executes the instructions so that the electronic device executes the method according to any one of claims 1-9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images