Audio processing method and electronic device

Abstract

This disclosure provides an audio processing method and an electronic device. The method is applied to a first device and includes: obtaining first audio data from a second device; processing the first audio data so that a first target device simultaneously plays the first audio data, the first target device including at least two of the first device and a third device; and establishing a communication connection between the first device and the third device, wherein the pickup ranges of the first device and the third device at least partially overlap.

Description

Technical Field

[0001] This disclosure relates to the field of audio signal processing technology, specifically to an audio processing method and an electronic device. Background Technology

[0002] In multi-device collaborative playback scenarios, overlapping sound pickup ranges between devices can easily cause acoustic feedback howling, and asynchronous playback can lead to auditory confusion, affecting the user experience. Summary of the Invention

[0003] In view of the above problems, this disclosure provides an audio processing method and an electronic device.

[0004] This disclosure provides an audio processing method applied to a first device, comprising: obtaining first audio data from a second device; processing the first audio data such that a first target device simultaneously plays the first audio data, the first target device including at least two of the first device and a third device; and establishing a communication connection between the first device and the third device, wherein the pickup ranges of the first device and the third device at least partially overlap.

[0005] According to embodiments of this disclosure, the method further includes: in response to the establishment of a first connection relationship between the first device and the third device, determining state information of the first device and the third device; if the state information indicates that the first device is in a first state, processing the first audio data so that the first target device simultaneously plays the first audio data.

[0006] According to embodiments of this disclosure, the method further includes: if the state information indicates that the first device is in a second state, suppressing the first audio data.

[0007] According to embodiments of this disclosure, processing the first audio data includes: if the first target device includes multiple third devices, simultaneously sending the first audio data to the third devices so that the multiple third devices play the first audio data simultaneously; if the first target device includes a first device and a third device, sending the first audio data to the third devices; determining playback delay information based on the transmission delay between the first device and the third devices; and playing the first audio data based on the playback delay information so that the first device and the third devices play the first audio data simultaneously.

[0008] According to embodiments of this disclosure, the method further includes: obtaining a first audio playback range of a first device and a second audio playback range of a third device; obtaining a target playback area, the target playback area being a spatial area expected to be covered by audio; and determining a first target device from the first device and the third device based on a first degree of overlap between the first audio playback range and the target playback area, and a second degree of overlap between the second audio playback range and the target playback area.

[0009] According to embodiments of this disclosure, the method further includes: if the state information indicates that the first device is in a first state, obtaining second audio data from the third device based on a first connection relationship; determining target audio data based on the second audio data and the third audio data; the third audio data being the audio data collected by the first device; and sending the target audio data to the second device through a second connection relationship; the second connection relationship being established between the first device, the second device, and the third device.

[0010] According to embodiments of this disclosure, the method further includes: if the status information indicates that the first device is in a first state, obtaining second audio data from the third device based on the first connection relationship; updating the status information of the first device and the third device based on the second audio data and the third audio data; the third audio data is the audio data collected by the first device.

[0011] According to embodiments of this disclosure, updating the status information of a first device and a third device based on second audio data and third audio data includes: determining a first audio parameter based on the second audio data; determining a second audio parameter based on the third audio data; the audio parameter includes at least one of audio features and the orientation information of a target sound source relative to the device; and updating the status information of the first device and the third device based on a comparison result of the first audio parameter and the second audio parameter.

[0012] According to embodiments of this disclosure, determining the status information of the first device and the third device includes: in response to the establishment of a first connection relationship between the first device and the third device, determining the status information of one of the first device and the third device as a first state, and determining the status information of the remaining devices as a second state.

[0013] This disclosure also provides an electronic device, including: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the following method: obtaining first audio data from a second device; processing the first audio data such that a first target device simultaneously plays the first audio data, the first target device including at least two of a first device and a third device; and establishing a communication connection between the first device and the third device, wherein the pickup ranges of the first device and the third device at least partially overlap.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0015] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0016] Figure 1 This diagram illustrates an application scenario of an audio processing method and an electronic device according to embodiments of the present disclosure.

[0017] Figure 2 A flowchart illustrating an audio processing method according to an embodiment of the present disclosure is shown schematically.

[0018] Figure 3 A network topology diagram illustrating the receiving of first audio data according to an embodiment of the present disclosure is shown schematically.

[0019] Figure 4 A network topology diagram illustrating the transmission of target audio data according to an embodiment of the present disclosure is shown schematically.

[0020] Figure 5 A signaling flowchart of an audio processing method according to an embodiment of the present disclosure is illustrated schematically;

[0021] Figure 6 A schematic block diagram of an audio processing apparatus according to an embodiment of the present disclosure is shown; and

[0022] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing an audio processing method according to an embodiment of the present disclosure. Detailed Implementation

[0023] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0024] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0025] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0026] It should be noted that the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information in this disclosed technical solution comply with relevant laws and regulations, necessary confidentiality measures have been taken, and it does not violate public order and good morals. In this disclosed technical solution, user authorization or consent has been obtained before acquiring or collecting user personal information.

[0027] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0028] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0029] Conference rooms are typically equipped with fixed audio and video conferencing equipment, such as... Figure 1 As shown. However, due to the limitations of microphone technology, when the speaker is far from the conference equipment, the picked-up sound is prone to distortion and unclearness, seriously affecting the conference audio experience. In real-world scenarios, participants often bring their personal laptops into the conference room. If the laptop and the fixed equipment in the conference room are connected to the same remote conference at the same time, their microphones and speakers will work simultaneously, which can easily cause acoustic feedback and howling due to acoustic coupling, further degrading the conference quality.

[0030] In view of the above, this disclosure provides an audio processing method and an electronic device, which will be described below with reference to the accompanying drawings.

[0031] Figure 1 Figure 100 schematically illustrates an application scenario of an audio processing method and electronic device according to an embodiment of the present disclosure.

[0032] It is important to note that Figure 1 The examples shown are merely examples of scenarios in which the embodiments of this disclosure can be applied, to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments or scenarios.

[0033] like Figure 1 As shown, the application scenario 100 according to this embodiment may include one or more local participants 101. The local participants 101 may be in an acoustic environment where multiple devices work together, such as a conference room, lecture hall, home or vehicle space, for example in scenarios such as multi-person video conferencing, remote teaching, hybrid office or home entertainment.

[0034] Application scenario 100 may further include a first device 102, at least one third device 103, a network 104, and a remote second device 105. The network 104 serves as a medium for providing a communication link between the first device 102, the third device 103, and the second device 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables. For example, a user can use the first device 102 and the third device 103 to interact with the second device 105 through the network 104 to receive or send audio data, etc. The first device 102 may be a fixed audio terminal in a conference room, a smart speaker, a video conferencing system, or a projector, or a personal computing device such as a laptop, smartphone, or tablet. The third device 103 includes, but is not limited to, personal computing devices such as laptops, smartphones, tablets, smart headphones, or portable microphone arrays. The remote second device 105 includes, but is not limited to, terminal devices for remote participants, conference servers, cloud audio processing nodes, or enterprise communication gateways.

[0035] It should be noted that both the first device 102 and the third device 103 can be fixed devices or personal computing devices. Their roles in collaborative work are not fixed. They can be the main control device in the collaborative group or the auxiliary device. The specific role can be dynamically determined based on the device capabilities, real-time status information and preset selection rules.

[0036] The first device 102 and the third device 103 can establish a local communication connection through a first connection relationship (such as Wi-Fi Direct, Bluetooth, or UWB technology), and their pickup ranges at least partially overlap to achieve local collaborative work. The first device 102 and the remote second device 105 can establish a remote communication connection through a second connection relationship (such as the Internet, cellular network, or corporate intranet) to access the same remote audio session.

[0037] During operation, when the remote second device 105 sends remote audio data, the first device 102 can receive the audio data through a second connection and coordinate with the third device 103 through a first connection, enabling at least two of the devices to synchronously play the remote audio. When the local participant 101 speaks, the first device 102 and the third device 103 can dynamically determine the target acquisition device with the best sound pickup effect based on the interactive state information (e.g., the third device 103 uses its directional sound pickup beam to capture the speaker's voice), and can send the acquired target audio data to the remote second device 105 through the second connection. Simultaneously, audio output from non-target acquisition devices can be suppressed to eliminate acoustic feedback. Ultimately, the local participant 101 can obtain a synchronized and clear remote voice playback experience, and the remote participant can also obtain high-quality local voice acquired by the best sound pickup device, thereby significantly improving the audio communication quality and auditory experience in multi-device collaborative scenarios.

[0038] It should be understood that Figure 1 The number of local participants, the first device, the third device, and the remote second device is merely illustrative. Depending on implementation needs, there can be any number of participants and collaborating devices.

[0039] It should be noted that the audio processing method provided in this disclosure can generally be applied to at least one of the first device or the third device, and correspondingly, the audio processing apparatus provided in this disclosure can be disposed in the first device or the third device. The audio processing method provided in this disclosure can also be executed by a server or server cluster (such as a local collaborative management server or a cloud conferencing server) that is different from the first device, the third device, and the remote second device and is capable of communicating with the devices. Accordingly, the audio processing apparatus provided in this disclosure can also be disposed in such a server or server cluster.

[0040] The following will be based on Figure 1 The described scene, through Figures 2-5 The audio processing method according to the embodiments of this disclosure will be described in detail.

[0041] like Figure 2 As shown, the audio processing method of this embodiment is applied to the first device and may include S210~S220.

[0042] In S210, the first audio data from the second device is obtained.

[0043] The second device can be a remote device, such as the terminal device (e.g., personal computer, mobile phone, conference terminal, etc.) of a remote participant in the same web conference. The first device, as one of the local devices, can receive the audio data stream (i.e., the first audio data) from the second device via network communication protocols. This audio data may contain the voice information of the remote participant. In practical applications, the first device can be a personal computing device such as a laptop or tablet carried by the user, or it can be a fixed conference host in the conference room.

[0044] During the acquisition of the first audio data, preprocessing operations can be performed. For example, jitter buffering can be applied to the received audio packets to eliminate latency jitter caused by network transmission and ensure the continuity of audio playback. Simultaneously, the audio data can be decoded, converting it into a digital audio signal that can be processed by playback devices. Quality assessment can also be performed on the audio data, such as detecting packet loss, noise, etc., and triggering packet loss concealment or error recovery mechanisms when necessary to ensure call quality. Furthermore, the first device can prepare to copy or distribute the first audio data according to local collaborative playback needs, so that it can be sent to multiple playback devices simultaneously later.

[0045] In S220, the first audio data is processed so that the first target device plays the first audio data simultaneously. The first target device includes at least two of the first device and the third device. The first device and the third device are connected in communication, and the pickup ranges of the first device and the third device at least partially overlap.

[0046] The first and third devices can be local devices working collaboratively within the same physical space. For example, the first device could be a conference room host, and the third device could be a laptop or extended speaker carried by a participant in the same conference room; or, the first device could be a participant's laptop, and the third device could be a tablet computer carried by another participant in the conference room. The first and third devices can establish a connection via wireless communication (such as Bluetooth or Wi-Fi) or wired communication (such as USB or audio cable) to form a local collaborative network. Because the first and third devices are located in the same conference room, office, or other enclosed or semi-enclosed space, their microphone pickup ranges overlap at least partially, meaning both devices can simultaneously capture the voice signal of the same speaker; correspondingly, the speaker playback ranges of each device can also overlap, collectively covering the entire space.

[0047] It should be noted that there can be multiple third devices. That is, local collaborative devices are not limited to the first device and a single third device, but can include multiple third devices, such as multiple personal computing devices in a conference room, which can communicate and connect with each other.

[0048] The first device can process the first audio data. Specifically, after receiving the first audio data from the second device, the first device can first determine which devices belong to the first target device, that is, determine the specific range of devices involved in this collaborative playback. The first target device may include the first device itself, one or more third devices, or a combination of both. It should be noted that the first device itself may or may not belong to the first target device. For example, in some scenarios, the first device may only act as a master node for audio distribution and processing, without participating in playback itself. In this case, the first target device only includes at least two third devices. In other scenarios, the first device may also participate in playback and become part of the first target device. Then, the first device can send the first audio data to the third devices that belong to the first target device, so that the first target device can play the first audio data synchronously. For third devices that do not belong to the first target device, the first device may not send the first audio data to them. During this process, the third device always suppresses the remote audio signal (i.e., the first audio data) from the second device to effectively avoid complex echo and howling problems caused by multiple devices not playing remote audio simultaneously.

[0049] In some embodiments, multi-device frequency-band collaborative playback can be employed. That is, when the first and third devices simultaneously play the processed remote audio, the audio signals can be processed differently based on the frequency response characteristics and spatial location of their respective speakers. For example, the first device (conference room host) can focus on playing mid-low frequency components to provide fullness of sound, while the third device (participant's laptop) can focus on playing mid-high frequency components to provide clarity and directionality of sound. This not only improves the overall listening experience but also further disrupts the coherence of acoustic feedback, enhancing robustness against howling.

[0050] In other embodiments, gain control and equalization adjustment can also be applied to the first audio data. Based on the differences in speaker performance between the first target devices (e.g., laptop built-in speakers have lower power, conference room speakers have higher power, and desktop speakers have medium power), the first device can independently adjust the volume, equalize, and control the dynamic range of the audio data sent to different target devices. This ensures that the output volume of each device matches the user's position and the device's acoustic characteristics, thereby avoiding discomfort caused by a device being too loud or too weak. For example, the main conference room speaker can serve as the primary playback device, covering the entire conference room space; the laptop speakers can supplement it, providing clearer direct sound for users near the laptop; and the desktop speakers can provide an auxiliary sound field for participants in specific areas.

[0051] Exemplarily, in a multi-person video conferencing scenario, a user brings a laptop into a meeting room and opens the conferencing software to join a remote meeting. At this time, the meeting room host, as the first device, after obtaining the first audio data of the remote second device, can first determine which devices belong to the first target devices based on the currently available device list, the online status of each device, the acoustic performance, and the meeting scenario requirements. For example, it can select itself and the laptop of User A as the target devices for this collaborative playback, and exclude the desktop microphone temporarily unused in the corner of the meeting room from the target devices. Subsequently, the meeting room host can accurately distribute the first audio data to each first target device. Only the selected third device (User A's laptop) receives the audio data from the meeting room host and synchronously plays the remote sound with the meeting room host itself. Throughout the process, all third devices not only always perform suppression processing on the remote audio signal to ensure that no additional echo path is generated, but also do not upload the audio signal collected by themselves to the remote end. Since the third device does not upload the audio signal to the remote end, the generation of an echo path is fundamentally eliminated, effectively avoiding the occurrence of the acoustic feedback whistling phenomenon. Finally, the sound in the meeting environment becomes clear and pure, and the participants can clearly hear the sound details in the meeting without being disturbed, significantly improving the audio experience of the meeting.

[0052] It can be understood that by having the first device assume the core processing and control functions and accurately distributing the first audio data to each first target device, enabling the target devices to simultaneously play the first audio data, synchronous playback under multi-device linkage is achieved, effectively avoiding the acoustic feedback whistling problem that is likely to occur when multiple devices are turned on simultaneously, and ensuring the clarity and stability of the call. In addition, this processing method is independent of the specific unified communication conferencing software type, has good compatibility and universality, and can significantly improve the meeting audio experience without changing the user's usage habits.

[0053] In an embodiment of the present disclosure, the method further includes: in response to the first device establishing a first connection relationship with the third device, determining the status information of the first device and the third device; if the status information indicates that the first device is in the first state, processing the first audio data so that the first target devices simultaneously play the first audio data.

[0054] When a first device (such as conference room audio equipment) establishes a first connection with a third device (such as a participant's laptop), the status information of the first and third devices can be determined first. The first connection can be a local area network (LAN) connection between the devices; for example, a wireless network can be used to connect the conference room equipment and laptops to the same LAN, enabling information exchange, downlink signal synchronization, and uplink signal transmission role selection between devices. Status information characterizes the role of the devices in collaborative playback and can include types such as a first state (master device) and a second state (slave device).

[0055] If the status information indicates that the first device is in the first state (i.e., the master device), then the first device can assume the responsibility of receiving and distributing remote audio signals. Specifically, when a remote participant sends voice through the second device, the first device, as the master node, receives the remote audio signal (i.e., the first audio data), processes the audio data according to its own and its subordinate devices' capabilities and status, determines which of its own and its subordinate devices belong to the first target devices, and distributes the first audio data to each first target device so that each first target device can play the first audio data simultaneously.

[0056] Figure 3 A schematic diagram of a network topology for receiving first audio data according to an embodiment of the present disclosure is shown.

[0057] In one embodiment, such as Figure 3 As shown, the second device 105 can send the first audio data to the repeater 106, and the first device 102 receives the first audio data from the repeater 106. The repeater 106, acting as a relay node for audio data, can receive the audio stream from the remote second device 105 and send it to the master control device (i.e., the first device 102) in the local network. In practical applications, the repeater 106 can be a standalone hardware device (such as a conference host or audio processor) or a software service deployed in the local network. After obtaining the first audio data through the repeater 106, the first device 102 can determine the first target device (which may include multiple third devices 103) and send the first audio data to each first target device for playback synchronization processing.

[0058] The master control device distributes the processed first audio data to each first target device. The distribution method can be flexibly selected according to the device connection type. For example, for first target devices connected via a local area network, the master control device can use multicast technology to send the audio stream to multiple devices simultaneously, reducing network bandwidth consumption and the processing burden on the master control device. As another example, for first target devices connected via point-to-point methods such as Bluetooth, the master control device can use unicast to send the data sequentially, adapting it according to the transmission characteristics of each device. Yet another example is that the master control device can adopt a hierarchical distribution strategy, that is, first sending the audio data to a repeater, and then the repeater forwards it to each slave device. This method is suitable for scenarios with a large number of devices and complex network topologies.

[0059] For example, in a multi-person office meeting scenario, participants bring their laptops into the meeting room, and the laptops automatically establish a wireless LAN connection with the meeting room's audio equipment. Based on hardware capabilities and power status, the system designates the meeting room audio equipment as the master device (i.e., the meeting room audio equipment is in the first state) and the laptop as the slave device. When a remote participant speaks, the meeting room audio equipment receives the remote audio signal. After determining that the primary target devices are the meeting room audio equipment and the laptop, it synchronously distributes the audio stream to the laptop's speakers via the LAN, and both devices simultaneously play the remote sound.

[0060] Understandably, by determining the master and slave devices based on status information, and using the master device as the core scheduling node to precisely synchronize the playback of multiple devices, the simultaneous playback of multiple devices can be achieved while effectively eliminating howling interference and significantly improving the quality of collaborative playback.

[0061] In embodiments of this disclosure, the method further includes: if the state information indicates that the first device is in a second state, suppressing the first audio data.

[0062] Once the first device and the third device establish a first connection, the roles of each device can be determined based on the status information. If the first device is determined to be in the second state (i.e., a slave device), it will suppress the remote audio by default. The slave device only acts as an auxiliary device; if it is determined to be the first target device, it receives the first audio data sent by the master device and plays it simultaneously with the master device. If it is not determined to be the first target device, its speaker will not play remote sound.

[0063] In one embodiment, the suppression process may include muting the first audio data. That is, the first audio data received by the slave device from the second device can be directly zeroed out or truncated and not sent to the speaker for playback. The muting process is simple to implement, consumes few resources, and can block the slave device from directly playing remote sounds that have not been processed by the master device, thus eliminating the risk of feedback at the source.

[0064] In another embodiment, the suppression processing may include attenuating the first audio data. The slave device may reduce the gain of the first audio data from the second device to below a preset threshold, for example, by 20 decibels or more, so that its playback volume is much lower than that of the master device, and almost imperceptible in the listening area. The attenuation processing can preserve the slave device's ability to quickly resume playback in emergency situations (such as when the master device loses connection), while the extremely low volume will not cause significant interference with the playback sound of the master device.

[0065] In another embodiment, the suppression process may include frequency domain filtering. The slave device can analyze the first audio data from the second device, identify frequency bands (such as mid-to-high frequency bands) that are prone to feedback, and attenuate or filter out only those frequency bands while preserving the faint playback of other frequency bands. This filtering method can suppress feedback while retaining a certain level of audio monitoring capability, making it suitable for scenarios where users want the slave device to provide personal monitoring without interfering with the main sound field.

[0066] Understandably, by suppressing the audio of slave devices, the problems of howling and sound field interference caused by asynchronous playback of distant sounds when multiple devices coexist in close proximity are solved.

[0067] Based on the above embodiments, in this embodiment, determining the status information of the first device and the third device includes: in response to the establishment of a first connection relationship between the first device and the third device, determining the status information of one of the first device and the third device as a first state, and determining the status information of the remaining devices as a second state.

[0068] In response to the establishment of a first connection between the first device and the third device, one of the two devices can be selected as the master device, and its status information is determined as the first state. Simultaneously, the status information of the remaining devices is determined as the second state, i.e., slave devices. The master device can undertake the responsibility of receiving, processing, and distributing remote audio signals, while the slave devices can, by default, suppress remote audio and only participate in coordinated playback under the scheduling of the master device. The method for determining the master device can be flexibly implemented according to various strategies.

[0069] For example, the master device can be determined based on the hardware capabilities of the equipment. For instance, conference room audio equipment typically has stronger audio processing capabilities (such as echo cancellation, beamforming, and multi-channel encoding / decoding) and can be preferentially identified as the master device in the first state; while laptops have relatively limited audio processing capabilities and can be identified as slave devices in the second state.

[0070] For example, the master device can be determined based on the power status of the device. For instance, a device connected to a power source can be preferentially identified as the master device to prevent battery-powered devices from running out of power prematurely due to continuous processing of audio signals; a battery-powered laptop can be identified as a slave device, only undertaking basic playback and sound pickup tasks.

[0071] For example, the determination of the master control device can also be based on the device's role in the meeting. For instance, if the conference room audio equipment is a permanently installed device, it can be designated as the master control device by default; if a participant's laptop is a temporarily connected device, it can be designated as a slave device. Alternatively, users can manually specify the master control device through the configuration interface to meet personalized needs in specific scenarios.

[0072] For example, the master device can also be determined based on device priority. In a local area network (LAN) environment, priorities can be pre-set for different devices, such as setting the conference machine's priority to the highest. When the first device establishes a connection with the third device, the priorities of each device can be automatically detected, and the conference machine with the highest priority can be identified as the first state, making it the initial master device responsible for coordinating the reception, processing, and distribution of remote audio signals. The status information of other devices is then identified as the second state, acting as subordinate devices and suppressing remote audio by default. Alternatively, the initiator of the LAN can be designated as the initial master device. That is, when a device initiates the establishment of a LAN for collaborative audio processing during conferencing, the initiating device can automatically become the master device, and its status information is identified as the first state. The status information of subsequently connected devices can then be identified as the second state.

[0073] It should be noted that the aforementioned status information is not static. During the meeting, the master control device can be changed based on actual needs and changes in device status, either through preset rules or manual user operation. For example, if the original master control device malfunctions, has insufficient power, or if the user wishes to replace the master control device for better audio processing, the system can re-determine the master control device according to a new strategy and update the relevant status information, thereby ensuring the smooth operation of collaborative audio processing during the meeting.

[0074] Understandably, by assigning master and slave roles, the responsibilities of each device in collaborative work are clearly defined, effectively avoiding conflicts caused by multiple devices processing audio signals simultaneously, thus laying a good foundation for synchronous audio distribution and collaborative playback.

[0075] Based on the above embodiments, in this embodiment, the processing of the first audio data includes: if the first target device includes multiple third devices, the first audio data is simultaneously sent to the third devices so that the multiple third devices play the first audio data simultaneously; if the first target device includes a first device and a third device, the first audio data is sent to the third devices; based on the transmission delay between the first device and the third devices, playback delay information is determined; based on the playback delay information, the first audio data is played so that the first device and the third devices play the first audio data simultaneously.

[0076] If the first target device includes multiple third devices (i.e., multiple slave devices are selected to participate in playback, while the master device itself does not participate in playback), the master device can simultaneously send the first audio data to these third devices, so that multiple third devices can play the first audio data at the same time. Simultaneous transmission can be achieved using multicast technology, that is, transmitting the audio data to multiple target devices simultaneously through a local area network, thereby reducing the transmission burden and network bandwidth consumption of the master device.

[0077] If the first target device includes a first device (the master device itself) and a third device (the slave device), the master device can send the first audio data to the third device while simultaneously preparing to play it itself. In this case, due to the physical distance and network transmission path between the first and third devices, the first audio data will experience a certain transmission delay from the master device to the slave device. If the master device immediately plays its own audio, while the slave device only starts playing after receiving the data, there will be a time difference between the sounds played by the two devices, causing the listener to hear two sounds arriving at different times, affecting the auditory experience. Therefore, the master device can coordinate the timing of its own playback with that of the third device.

[0078] First, the master control device can measure the transmission delay with the third device. Transmission delay measurement can be achieved in several ways. For example, the master control device can send a probe packet to the third device, which immediately responds upon receiving it. The master control device estimates the one-way transmission delay based on half the round-trip time. Alternatively, the master control device can use a network time protocol to synchronize the time between devices, calculating the transmission delay based on the timestamp difference. Another example is that the master control device can embed a transmission timestamp into the audio data; the third device, upon receiving the data, calculates the transmission delay based on the difference between its local time and the timestamp. Based on the measured transmission delay, the master control device can determine playback delay information, which instructs each device how long it should wait before starting playback to ensure that all speakers of the first target devices emit sound at the same time. The determination of playback delay information can employ a maximum delay alignment strategy: the master control device measures the transmission delay of all first target devices, takes the maximum value as the baseline delay, and then calculates a compensation delay for each device—the baseline delay minus the device's transmission delay. Devices with smaller transmission delays can wait for the compensation delay before starting playback, while the device with the largest transmission delay plays immediately.

[0079] Based on the determined playback delay information, each first target device can execute playback according to its own waiting time. Specifically, for the master control device itself, if its transmission delay (i.e., the inherent delay of its own audio processing link) is less than the baseline delay, it needs to wait for the compensation delay before starting playback; if its delay is equal to the baseline delay, it plays immediately. For the third device, when sending the first audio data, the master control device can attach playback delay information to the data packet, instructing the third device to wait a specified time after receiving the data before starting playback. After receiving the audio data, the third device can start an internal timer and start playback after the specified delay. Ultimately, all first target devices can output sound at the same time, achieving auditory synchronization.

[0080] In other embodiments, the master device may employ an advance transmission strategy to compensate for transmission delays. For example, the master device may send the first audio data to the third device an amount of time delay in advance, so that the master device itself begins playing the data at the same time the third device receives it, thereby achieving natural synchronization.

[0081] For example, in a conference room scenario, the master control device (conference audio equipment) first identifies the first target devices as itself and two laptops (third device A and third device B). Then, the master control device measures the transmission latency: 20 milliseconds with device A and 35 milliseconds with device B. Taking the maximum latency of 35 milliseconds as the baseline latency, the master control device determines the playback latency information as follows: device B's compensated latency is 0 (immediate playback), the master control device's own compensated latency is 15 milliseconds (35-20), and device A's compensated latency is 15 milliseconds (35-20). Subsequently, the master control device can send the first audio data to devices A and B, attaching their respective compensated latency information to the data packets. The master control device waits 15 milliseconds before starting playback, device A waits 15 milliseconds after receiving the data before starting playback, and device B starts playing immediately after receiving the data. Ultimately, the speakers of all three devices emit sound simultaneously.

[0082] Understandably, synchronous playback based on transmission delay measurement effectively solves the time alignment problem in multi-device collaborative playback, ensuring the consistency and continuity of remote sound across multiple devices, and significantly improving the audio experience in multi-device collaborative scenarios.

[0083] In embodiments of this disclosure, the method further includes: obtaining a first audio playback range of a first device and a second audio playback range of a third device; obtaining a target playback area, wherein the target playback area is a spatial area that is expected to be covered by audio; and determining a first target device from the first device and the third device based on a first degree of overlap between the first audio playback range and the target playback area, and a second degree of overlap between the second audio playback range and the target playback area.

[0084] The first audio playback range of the first device and the second audio playback range of the third device can be used to characterize the area in space that the sound emitted by the speakers of each device can effectively cover. This area can be obtained in a variety of ways.

[0085] In one embodiment, acoustic modeling can be performed based on the device's location information and speaker parameters. For example, the precise coordinates of each device in the conference room can be obtained using indoor positioning technology, and combined with the device's speaker directivity, power parameters, and the conference room's spatial structure (such as wall reflections, sound-absorbing materials, etc.), the sound pressure level distribution of each device's speaker at various locations in the space can be calculated, and the area with a sound pressure level higher than a preset threshold can be determined as the device's audio playback range.

[0086] In another embodiment, the playback range of the device can be obtained through pre-measurement. For example, when deployed in a conference room, professional acoustic measurement instruments can be used to calibrate the actual playback effect of each device at different locations, and the measurement data can be stored as an inherent attribute of the device. When the device is connected to the system, its pre-stored playback range information can be automatically loaded.

[0087] In another embodiment, the playback range can be set manually by the user. For example, the meeting host can use a visual interface to mark the location and coverage direction of each device on a floor plan of the meeting room, and the system can generate the corresponding playback range based on the marked information.

[0088] The target playback area is the spatial region where audio coverage is desired, i.e., the area where distant sound should be clearly transmitted. The target playback area can be dynamically determined based on the distribution of participants. For example, cameras in the meeting room can capture images, and human detection and posture recognition algorithms can be used to determine the seating positions of participants. These positions can then be aggregated into one or more target areas to ensure that all participants are covered by audio. Alternatively, the target playback area can be manually specified by the user; the meeting host can delineate the area to be covered on the meeting room floor plan, such as the conference table area, the projection screen area, or the area where all participants are located.

[0089] After obtaining the audio playback range and target playback area of ​​each device, the first degree of overlap between the first audio playback range and the target playback area, and the second degree of overlap between the second audio playback range and the target playback area, can be calculated. The degree of overlap can be quantified as the area ratio, volume ratio, or weighted score of the overlapping portion of the playback range and the target area. Based on the degree of overlap calculation results, the first target device can be determined from the first device and the third device.

[0090] For example, devices can be sorted from highest to lowest overlap, and the devices with the highest overlap can be selected as the first target devices. For instance, in scenarios where the number of playback devices needs to be controlled (e.g., to avoid interference from too many devices playing simultaneously), only the two devices with the highest overlap can be selected for playback. Alternatively, an overlap threshold can be set, and devices with overlap exceeding the threshold can be identified as the first target devices. For example, only devices that can cover more than 50% of the target area can be selected for playback, thus ensuring that each selected device can make an effective contribution to the target area. Furthermore, when multiple devices overlap in coverage of the same area, devices with better acoustic characteristics (e.g., lower distortion, flatter frequency response) or devices that coordinate more stably with the main control device can be prioritized.

[0091] For example, in a conference room scenario, the conference room audio equipment (first device) is installed at the front of the room, and its first audio playback range mainly covers the front row area of ​​the conference room. Two participants' laptops (third device A and third device B) are placed in the middle and back rows of the conference table, respectively, and their playback ranges cover the middle and back rows. The system identifies the participants' distribution in the middle and back rows through a camera, thus determining the middle and back rows as the target playback areas. At this time, by calculating the overlap, it is found that the first overlap between the first device and the target area is only 30% (mainly covering the front row, with limited overlap with the target area), the second overlap between third device A and the target area is 80%, and the second overlap between third device B and the target area is 85%. Based on the overlap results, the system determines third device A and third device B as the first target devices to participate in the synchronous playback of remote audio, while the first device is not selected due to insufficient coverage and only acts as the master control device responsible for audio reception and distribution.

[0092] Understandably, by evaluating the coverage capabilities of each device for the target playback area and selecting the best device to participate in the playback, the system ensures that the distant sound can accurately cover the area where the participants are located, avoiding sound field interference and resource waste caused by blindly activating all devices, and significantly improving the audio coverage efficiency and quality in multi-device collaborative scenarios.

[0093] In embodiments of this disclosure, the method further includes: if the state information indicates that the first device is in a first state, obtaining second audio data from the third device based on a first connection relationship; determining target audio data based on the second audio data and the third audio data; the third audio data being the audio data collected by the first device; and sending the target audio data to the second device through a second connection relationship; the second connection relationship being established between the first device, the second device, and the third device.

[0094] When the first device is in its first state, acting as the master device, it can not only distribute and play remote audio but also process local audio and upload it to the remote end. Specifically, the master device can collect ambient sound through its built-in or external microphone to obtain third audio data. Simultaneously, the third device (slave device) can also collect ambient sound through its microphone and send the collected second audio data to the first device via a first connection (local network connection). After obtaining audio data from multiple devices, the master device can determine the target audio data based on the second and third audio data. In one embodiment, an energy comparison strategy can be used to determine the target audio data. That is, the master device can analyze the energy levels of the audio signals collected by each device and select the signal with the strongest energy and highest signal-to-noise ratio as the target audio data. Since the device closer to the participant usually collects a larger signal energy, this strategy can automatically select the device closest to the speaker for uplink transmission, achieving an intelligent sound pickup effect of "whoever is closest uploads."

[0095] In another embodiment, a signal fusion strategy can be employed. That is, the master control device can align, weight, and mix audio signals collected by multiple devices to form a fused audio stream. During the fusion process, signals with higher signal-to-noise ratios can be assigned greater weights, while the correlation of multiple signals can be used to suppress noise and improve the clarity of uplink speech.

[0096] In another embodiment, a sound source localization strategy can be adopted. That is, the main control device can determine the azimuth angle of the sound source by analyzing the beamforming results of the microphone arrays of each device, select the device whose sound source falls within its pickup beam as the main pickup device, and determine the audio data collected by it as the target audio data.

[0097] Before determining the target audio data, the master device can first decide whether it is necessary to determine the target audio data. For example, when a remote device outputs sound (i.e., someone is speaking), the master device can distribute audio to the slave device as needed. During audio playback, the slave device's microphone will pick up the remote audio and send it to the master device. Since the sound picked up by the slave device already originates from the remote location, uploading it back would inevitably cause feedback. Therefore, in this situation, the master device can determine that there is no need to confirm the target audio data, and the sound picked up by the slave device will not be uploaded to the remote location, thus avoiding feedback. Only when the master device determines that the speaker is from the local location will it determine the target audio data and send it to the remote location. After determining the target audio data, the master device can send it to the second device (the remote device) through a second connection. The second connection is a remote connection established between the first, second, and third devices, and can be an internet connection to support remote audio sessions.

[0098] To prevent uplink-captured local audio from being picked up again by the same or adjacent devices during downlink playback, creating echoes or feedback, after identifying the primary uplink microphone, other devices on the local network can be instructed to mute on the downlink. Specifically, when a device is selected as the primary microphone (its captured audio is uploaded to a remote location), other devices can suppress (mute or attenuate) the local speaker's voice on the downlink, preventing it from being picked up and played back by other devices, thus effectively blocking the positive feedback loop of feedback. Simultaneously, the primary microphone can play remote audio normally (if it is included in the first target device), while its own remote audio can be processed using echo cancellation technology to prevent it from being picked up by its own microphone.

[0099] Figure 4 A network topology diagram illustrating the transmission of target audio data according to an embodiment of the present disclosure is shown schematically.

[0100] like Figure 4As shown, the network topology includes a second device 105 (remote device), a first device 102 (master device), multiple third devices (slave devices) 103, and a repeater 106. The second device 105 establishes a remote audio session with the first device 102 and the third devices 103 through a second connection (Internet connection). Simultaneously, the first device 102 can form a local collaborative network with the multiple third devices 103 through a first connection (local network connection). When a local participant speaks, the first device 102 and the third devices 103 respectively collect third and second audio data. The first device 102, as the master node, can collect audio signals from each device and select the optimal audio signal as the target audio data through energy comparison or sound source localization strategies. This signal is then sent to the repeater 106 through the second connection, and from there to the second device 105. Simultaneously, the first device 102 can control non-master audio pickup devices in the local network to mute on the downlink to avoid feedback.

[0101] Figure 5 A signaling flowchart of an audio processing method according to an embodiment of the present disclosure is illustrated schematically.

[0102] like Figure 5 As shown, firstly, the system can automatically discover and identify available audio devices in the local network via ultrasound, Bluetooth, or network broadcasting. These include a first device as the master control device, a third device as a collaborative playback device, and a second device as a remote audio source. Secondly, a first connection can be established between the first and third devices to form a local collaborative network. Simultaneously, a second connection (such as an internet connection) can be established between the first, second, and third devices to access the same remote audio session. Then, the second device can send first audio data to the first device through the second connection, and the first device, acting as the master control device, receives this remote audio data. Simultaneously, the first device can send the acquired target audio data to the second device through the second connection. Based on this, each device can achieve high-quality, feedback-free remote communication through a client application.

[0103] Understandably, by determining the target audio data through the main control device and sending the data to the second device, intelligent sound pickup and feedback suppression are achieved in multi-device collaborative scenarios, ensuring that the local speaker's voice can be transmitted clearly and stably to the remote end.

[0104] In embodiments of this disclosure, the method further includes: if the status information indicates that the first device is in a first state, obtaining second audio data from the third device based on the first connection relationship; updating the status information of the first device and the third device based on the second audio data and the third audio data; the third audio data is the audio data collected by the first device.

[0105] When the first device is in the first state (master device), it can obtain second audio data from the third device (slave device) through the first connection relationship, and can also collect third audio data itself. Then, the master device can analyze the second and third audio data to evaluate the sound pickup quality of each device.

[0106] Based on the analysis results, the master control device can redefine the roles of each device in collaborative work and update the status information of the first and third devices. For example, if it is found that the sound pickup quality of a certain slave device is consistently better than that of the master control device, the roles of the master control device and the slave device can be swapped. That is, the status information of the first device, which was originally in the first state, is updated to the second state, and the status information of a certain third device, which was originally in the second state, is updated to the first state.

[0107] If the updated status information of each device indicates that the first device changes from the first state to the second state (i.e., the master device is downgraded to a slave device), and a third device is re-determined to the first state (i.e., upgraded to the master device), then the original first device can send the re-determined status information of each device to the third device, so that the third device with the status information of the first state can take over the master control responsibilities and send the collected target audio data to the second device (remote device).

[0108] In other embodiments, when the master device malfunctions (e.g., power outage, network outage, system crash), the system can automatically trigger a master-slave switchover process. After a slave device detects the master device is offline via a heartbeat detection mechanism, it can select a new master device (its status information is updated from the second state to the first state) to take over the task, based on a preset alternative order or through renegotiation. Alternatively, when a user moves with their laptop, the device's location and network status may change. The system can monitor device status in real time, and when it detects that a slave device has better network quality or location conditions than the current master device, it can proactively initiate a master-slave role switch, promoting the better device to master, i.e., updating its status information to the first state, thereby improving overall collaboration.

[0109] For example, in an office meeting scenario, the initial state designates the conference room audio equipment (Device A) as the master device (first state) and the laptop (Device B) as the slave device (second state). During the meeting, a speaker moves from the front row to the back row, closer to Device B. Device A analyzes the collected audio data and finds that Device B's audio signal energy is significantly higher than its own, and Device B has a better signal-to-noise ratio. Based on this, Device A updates its state information, changing its own state to second state and Device B's state to first state. Then, Device A sends the updated state information to Device B, notifying it to take over the master control responsibilities. After receiving the state switching instruction, Device B begins to send the collected second audio data to the remote participants through the second connection relationship.

[0110] Understandably, by introducing a status information update mechanism based on real-time audio analysis, the system ensures that the device with better sound pickup always assumes the main control responsibility, thereby significantly improving the stability and intelligence of the conference audio experience.

[0111] Based on the above embodiments, in this embodiment, updating the status information of the first device and the third device based on the second audio data and the third audio data includes: determining the first audio parameter based on the second audio data; determining the second audio parameter based on the third audio data; the audio parameter includes at least one of audio features and the directional information of the target sound source relative to the device; and updating the status information of the first device and the third device based on the comparison result of the first audio parameter and the second audio parameter.

[0112] The main control device can determine the first audio parameters based on the second audio data obtained from the third device. Simultaneously, it can determine the second audio parameters based on the third audio data it collects itself. The audio parameters may include at least one of audio characteristics and the directional information of the target sound source relative to the device, reflecting the device's sound pickup quality from different dimensions.

[0113] For example, audio features may include indicators such as signal energy, signal-to-noise ratio (SNR), and speech intelligibility. Among them, signal energy reflects the sound intensity captured by the device; the higher the energy, the closer the device is to the sound source. The SNR reflects the ratio of effective speech to background noise in the signal; the higher the SNR, the better the sound pickup quality. Speech intelligibility can comprehensively evaluate the intelligibility of the audio.

[0114] For example, audio parameters may include the azimuth information of the target sound source relative to the device. The master device can analyze the audio signals collected by the microphone arrays of each device and use direction-of-arrival (DOA) estimation technology to calculate the azimuth angle of the sound source relative to each device, including the horizontal and vertical angles. The azimuth information can be used to determine whether the sound source is within the main pickup beam range of the device, thereby evaluating the device's ability to pick up the sound source.

[0115] It should be noted that audio parameters can include both audio characteristics and location information, which corroborate each other to improve the accuracy of the assessment. For example, when the signal energy of two devices is similar, location information can be used to determine which device is more directly facing the sound source, thus making a more reasonable state update decision.

[0116] The master control device can compare the first audio parameters with the second audio parameters and update the status information of the first and third devices based on the comparison results. When the comparison results show that the pickup quality of the third device is significantly better than that of the first device—for example, the signal energy collected by the third device is consistently higher than that of the first device, the signal-to-noise ratio is higher, and the sound source is located in the center region of its main pickup beam—the master control device can update the status of the first device from the first state to the second state, and update the status of the third device from the second state back to the first state, thus achieving dynamic switching between master and slave roles. When the comparison results show that the pickup quality of the first device is still better than that of all third devices, the master control device can maintain its current state. When the comparison results show that the pickup quality of multiple devices is similar, the master control device can maintain its current state to avoid frequent state switching that could cause system jitter.

[0117] Understandably, state updates are performed based on quantitative comparison of audio parameters, thereby achieving objective and intelligent decision-making for master-slave role switching and avoiding the uncertainty of subjective judgment.

[0118] For example, in a multi-user video conferencing scenario, multiple users bring laptops into the conference room and join the remote meeting by opening conferencing software. The conference room host acts as the first device, and the multiple laptops act as third devices. The two establish a first connection relationship through a wireless LAN, forming a local collaborative network. When a remote participant speaks through a second device, the conference room host, as the master device (first state), first receives the first audio data from the second device. The master device can determine the first target device based on the audio playback range and target playback area of ​​the local device. For example, it can select itself and user A's laptop as the target devices for this collaborative playback. Subsequently, the conference room host can accurately distribute the first audio data to each first target device through the first connection relationship. After receiving the audio data from the conference room host, the laptop selected as the first target device (third device) can synchronously play the remote sound with the conference room host according to the playback delay information sent by the master device. For other subordinate devices not selected as first target devices (such as user B's laptop), they always suppress the remote audio signal, that is, their speakers do not play the remote sound, thereby avoiding the generation of additional echo paths. Simultaneously, these subordinate devices do not upload their collected audio to the remote end, thus avoiding acoustic feedback loops and effectively eliminating the risk of howling. Through this selective playback mechanism, the sound of remote participants can be clearly and synchronously covered to the area where all participants are located, while avoiding sound field interference caused by multiple devices not playing simultaneously. When a local participant speaks, the microphones of the conference room host (master device) and each subordinate device can simultaneously capture the local speaker's voice. The master device collects its own collected third audio data, as well as second audio data obtained from each subordinate device through the first connection relationship. The master device can analyze the multi-channel audio data, determine the target audio data, and send the target audio data to the remote second device through the second connection relationship. At the same time, the subordinate devices do not upload their collected local audio to the remote end, thus avoiding echo paths. During this process, the device status information can be updated; that is, the original master device's status information can change from the first state to the second state, and a subordinate device can change from the second state to the first state. Through this mechanism, in multi-device collaborative meeting scenarios, remote audio is accurately and synchronously played to the area where participants are located, while the voice of the speaker at the near end is only uploaded to the remote end by the main control device, thus effectively eliminating the risk of feedback. Users can obtain a clear, stable, and feedback-free meeting audio experience without manually switching devices or adjusting the volume. Figure 6 A block diagram of an audio processing apparatus according to an embodiment of the present disclosure is shown schematically.

[0119] like Figure 6 As shown, the audio processing device 600 includes an acquisition module 610 and a playback module 620.

[0120] According to some embodiments of this disclosure, the audio processing apparatus 600 can be used to implement the reference. Figures 2-5 The audio processing method described according to embodiments of the present disclosure.

[0121] The acquisition module 610 can perform, for example, operation S210, to acquire first audio data from the second device.

[0122] The playback module 620 can perform, for example, operation S220, to process the first audio data so that the first target device simultaneously plays the first audio data, the first target device including at least two of the first device and the third device; the first device and the third device are communicatively connected, and the pickup ranges of the first device and the third device at least partially overlap.

[0123] For example, any plurality of the acquisition module 610 and playback module 620 can be combined into one module, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the acquisition module 610 and playback module 620 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of the three implementation methods: software, hardware, and firmware. Alternatively, at least one of the acquisition module 610 and playback module 620 can be at least partially implemented as a computer program module, which, when run, can perform corresponding functions.

[0124] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing an audio processing method according to an embodiment of the present disclosure.

[0125] like Figure 7As shown, an electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage portion 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0126] RAM 703 stores various programs and data required for the operation of electronic device 700. Processor 701, ROM 702, and RAM 703 are interconnected via bus 704. Processor 701 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 702 and / or RAM 703. It should be noted that programs may also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in one or more memories.

[0127] According to embodiments of this disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to a bus 704. The electronic device 700 may also include one or more of the following components connected to the I / O interface 705: an input section 706 including target hardware, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0128] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0129] According to embodiments of this disclosure, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 702 and / or RAM 703 and / or one or more memories other than ROM 702 and RAM 703 described above.

[0130] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to enable the computer system to implement the audio processing methods provided in the embodiments of this disclosure.

[0131] When the computer program is executed by the processor 701, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0132] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 709, and / or installed from a removable medium 711. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0133] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0134] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on a user's computing device, partially on a user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0135] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0136] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0137] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. An audio processing method, applied to a first device, comprising: Obtain the first audio data from the second device; The first audio data is processed so that a first target device plays the first audio data simultaneously, the first target device including at least two of the first device and the third device; the first device and the third device are communicatively connected, and the pickup ranges of the first device and the third device at least partially overlap.

2. The method according to claim 1, further comprising: In response to the establishment of a first connection relationship between the first device and the third device, the status information of the first device and the third device is determined; If the status information indicates that the first device is in a first state, the first audio data is processed so that the first target device plays the first audio data simultaneously.

3. The method according to claim 2, further comprising: If the status information indicates that the first device is in the second state, the first audio data is suppressed.

4. The method according to claim 1, wherein processing the first audio data includes: If the first target device includes multiple third devices, the first audio data is simultaneously sent to the third devices so that the multiple third devices play the first audio data simultaneously. If the first target device includes a first device and a third device, the first audio data is sent to the third device; Based on the transmission delay between the first device and the third device, the playback delay information is determined; Based on the playback delay information, the first audio data is played so that the first device and the third device play the first audio data simultaneously.

5. The method according to claim 1, further comprising: Obtain the first audio playback range of the first device and the second audio playback range of the third device; Obtain the target playback area, which is the spatial area that is expected to be covered by audio. The first target device is determined from the first device and the third device based on the first degree of overlap between the first audio playback range and the target playback area, and the second degree of overlap between the second audio playback range and the target playback area.

6. The method according to claim 2, further comprising: If the status information indicates that the first device is in a first state, then based on the first connection relationship, the second audio data from the third device is obtained; Based on the second and third audio data, the target audio data is determined; The third audio data is the audio data collected by the first device; The target audio data is sent to the second device through a second connection relationship; the second connection relationship is established between the first device, the second device, and the third device.

7. The method according to claim 2, further comprising: If the status information indicates that the first device is in a first state, then based on the first connection relationship, the second audio data from the third device is obtained; Based on the second and third audio data, update the status information of the first and third devices; the third audio data is the audio data collected by the first device.

8. The method according to claim 7, wherein updating the status information of the first device and the third device based on the second audio data and the third audio data comprises: Based on the second audio data, the first audio parameters are determined; Based on the third audio data, the second audio parameters are determined; The audio parameters include at least one of audio features and the location information of the target sound source relative to the device; Based on the comparison results of the first audio parameter and the second audio parameter, the status information of the first device and the third device is updated.

9. The method according to claim 2, wherein determining the status information of the first device and the third device includes: In response to the establishment of a first connection between the first device and the third device, the status information of one of the first device and the third device is determined as a first state, and the status information of the remaining devices is determined as a second state.

10. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors perform the following method: Obtain first audio data from a second device; process the first audio data so that a first target device simultaneously plays the first audio data, the first target device including at least two of a first device and a third device; the first device and the third device are communicatively connected, and the pickup ranges of the first device and the third device at least partially overlap.

Images