Audio processing method and device, electronic equipment and storage medium

Abstract

The invention provides an audio processing method and device, electronic equipment and a storage medium, and the method is applied to audio equipment with an audio function, and comprises the steps: obtaining an initial audio in real time, and determining each tone feature included in the initial audio and a first signal intensity corresponding to each tone feature according to the initial audio; receiving each timbre feature sent by each other device and a second signal intensity corresponding to each timbre feature, and determining a target timbre feature from each timbre feature according to the first signal intensity corresponding to the timbre feature and each second signal intensity for each timbre feature; and processing the initial audio according to the target timbre feature to obtain a target audio corresponding to the target timbre feature, and sending the target audio to other devices, so that the other devices play the target audio with the same signal strength as the audio device. Through the technical scheme of the invention, the robustness and stability of audio acquisition, processing and playing are improved, and the audio definition played by each device is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an audio processing method, device, electronic device and storage medium. Background Art

[0002] When in a space, multiple people can use computers or mobile phones together to conduct telephone conferences, voice calls or online meetings to achieve the purpose of online meetings.

[0003] Then, when a speaker is speaking, the microphones of devices at different locations will collect the speaker's speech content. The microphones that are far away from the speaker will have long-distance audio collection, which will affect the quality of audio collection. Since the collected audio will be played by the corresponding speakers, it will also lead to poor audio playback effects, making it difficult for participants to hear the meeting content clearly, affecting meeting efficiency. Summary of the invention

[0004] In view of this, the purpose of the present application is to propose an audio processing method, device, electronic device and storage medium to solve the problem that when multiple devices at different locations in the same space collect and play audio at the same time, the collected audio quality is poor due to the long receiving distance of some microphones, resulting in poor playback effect of the corresponding speakers.

[0005] Based on the above purpose, the present application provides an audio processing method, which is applied to an audio device with an audio function, and the method includes:

[0006] Acquire initial audio in real time, and determine, based on the initial audio, various timbre features included in the initial audio and first signal strengths corresponding to the various timbre features;

[0007] Receiving each timbre feature and the second signal strength corresponding to each timbre feature sent by each other device, and for each timbre feature, determining a target timbre feature from each timbre feature according to the first signal strength corresponding to the timbre feature and each second signal strength;

[0008] The initial audio is processed according to the target timbre feature to obtain a target audio corresponding to the target timbre feature, and the target audio is sent to each other device so that each other device plays the target audio with the same signal strength as the audio device.

[0009] Based on the above purpose, the present application also provides an audio processing device, which is configured in an audio device with an audio function, and the device includes:

[0010] The acquisition and processing module is used to acquire the initial audio in real time, and determine, according to the initial audio, each timbre feature included in the initial audio and the first signal strength corresponding to each timbre feature;

[0011] A target timbre feature determination module is used to receive each timbre feature and the second signal strength corresponding to each timbre feature sent by each other device, and for each timbre feature, determine the target timbre feature from each timbre feature according to the first signal strength and each second signal strength corresponding to the timbre feature;

[0012] The audio processing module is used to process the initial audio according to the target timbre feature to obtain the target audio corresponding to the target timbre feature, and send the target audio to each other device so that each other device plays the target audio with the same signal strength as the audio device.

[0013] Based on the above purpose, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, an audio processing method as provided in any embodiment of the present application is implemented.

[0014] Based on the above purpose, the present application also provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the audio processing method provided in any embodiment of the present application.

[0015] As can be seen from the above, the audio processing method provided by the present application obtains the initial audio in real time, determines the timbre features included in the initial audio and the first signal strength corresponding to each timbre feature according to the initial audio, so as to separate the timbre features, and receives the timbre features sent by other devices and the second signal strength corresponding to each timbre feature. For each timbre feature, according to the first signal strength corresponding to the timbre feature and each second signal strength, the target timbre feature is determined from each timbre feature, so as to determine the acquisition device corresponding to each timbre feature, so as to ensure the optimal audio acquisition method. Further, the initial audio is processed according to the target timbre feature to obtain the target audio corresponding to the target timbre feature, so as to ensure that the audio device only acquires the target audio and avoids interference from other timbre signals, and sends the target audio to other devices, so that each other device plays the target audio with the same signal strength as the audio device, thereby realizing audio collaborative processing of multiple devices, improving the robustness and stability of audio acquisition, processing and playback, and optimizing the resource allocation of audio processing through the strongest signal device responsibility system, dispersing the processing load, and improving the overall efficiency and audio clarity. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A flowchart of an audio processing method provided in an embodiment of the present application;

[0018] Figure 2 A flowchart of another audio processing method provided in an embodiment of the present application;

[0019] Figure 3 A schematic diagram of the structure of an audio processing device provided in an embodiment of the present application;

[0020] Figure 4 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0022] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0023] Figure 1 The flowchart of an audio processing method provided in an embodiment of the present application is applicable to the case where multiple devices are located in different positions in the same space to collect and play audio. The method can be configured in an electronic device, which can specifically be an audio device with audio function. Figure 1 As shown, the method may specifically include the following steps:

[0024] S110: Acquire initial audio in real time, and determine, according to the initial audio, various timbre features included in the initial audio and first signal strengths corresponding to the various timbre features.

[0025] The initial audio may be audio collected by a microphone of an audio device. To ensure audio quality, the initial audio may also be audio after anti-interference processing has been performed on the audio collected by the microphone. The initial audio may include audio of one or more speakers. The timbre feature is a unique feature of different sounds in the initial audio, which can be used to distinguish different speakers. The first signal strength is the sound signal strength of the portion of audio corresponding to the timbre feature determined by the audio device analysis.

[0026] Specifically, the initial audio can be acquired in real time through the microphone of the audio device or the microphone and the processing module. Then, the processing module performs timbre feature analysis on the initial audio, determines various timbre features contained in the initial audio, and calculates the signal strength corresponding to each timbre feature, which is the first signal strength of the timbre feature.

[0027] Based on the above example, since the initial audio may be audio after anti-interference processing, wherein the anti-interference processing may be echo cancellation processing, the initial audio may be acquired in real time in the following manner:

[0028] The current audio is collected in real time, and based on the echo cancellation algorithm, the audio played by the audio device is removed from the current audio to obtain the initial audio.

[0029] Among them, the echo cancellation algorithm is an algorithm used to solve the problem that the sound played by the speaker is picked up again by the microphone during a real-time call. The echo cancellation algorithm can be a least mean square algorithm (Least Mean Square, LMS), or other algorithms for eliminating echoes. LMS is an adaptive filtering algorithm that can be used for echo cancellation in real-time voice communication.

[0030] Specifically, the current audio can be collected in real time through the microphone of the audio device. The echo cancellation algorithm is used to perform echo cancellation processing on the current audio to process the microphone and speaker signals, and remove the audio played by the audio device from the current audio. This prevents the sound played by the speaker of the audio device from being picked up again by the microphone and repeatedly played to produce an echo. The current audio after the echo cancellation processing is the initial audio.

[0031] The echo cancellation algorithm can effectively eliminate the echo interference between the speaker and the microphone and improve the audio quality.

[0032] Based on the above example, since the initial audio may be audio after anti-interference processing, wherein the anti-interference processing may be anti-crosstalk processing, the initial audio may be acquired in real time in the following manner:

[0033] The current audio is collected in real time, and based on the phase delay algorithm and the audio cancellation algorithm, the audio played by other devices is removed from the current audio to obtain the initial audio.

[0034] Among them, the phase delay algorithm is an algorithm for adjusting the time axis of the audio signal, and the audio cancellation algorithm is an algorithm for eliminating redundant audio signals.

[0035] Specifically, the current audio is collected in real time, and based on a phase delay algorithm and an audio cancellation algorithm, the audio played by each other device is removed from the current audio to obtain the initial audio.

[0036] By using phase delay algorithm and audio cancellation algorithm, crosstalk and excess noise between devices can be reduced and audio clarity can be improved.

[0037] Optionally, the initial audio may be further processed for noise elimination to avoid interference from current sounds between devices, ambient sounds in the space, etc., and to update the initial audio to achieve better audio effects.

[0038] S120, receiving each timbre feature and the second signal strength corresponding to each timbre feature sent by each other device, and for each timbre feature, determining a target timbre feature from each timbre feature according to the first signal strength corresponding to the timbre feature and each second signal strength.

[0039] Among them, other devices are devices that are located in different positions in the same space as the audio device and collect and play audio together. For example, multiple users use audio devices and other devices to hold online conferences in the same conference room. The second signal strength is the signal strength of each timbre feature analyzed by each other device received by the audio device. The target timbre feature is the timbre feature that the audio device needs to collect separately later.

[0040] Specifically, since other devices also collect audio in the same space, each other device also extracts timbre features, determines the signal strength of each timbre feature, and sends it to the audio device. The audio device receives each timbre feature sent by other devices, and receives the signal strength corresponding to each timbre feature. Since the signal strength determined by other devices does not match the audio device, these signal strengths are used as second signal strengths. For each timbre feature, a comparison is made based on the first signal strength and each second signal strength corresponding to the timbre feature to determine whether the audio device is the best acquisition device for the timbre feature. Based on this, the target timbre feature can be determined from the timbre features.

[0041] Based on the above example, after determining each timbre feature and the first signal strength corresponding to each timbre feature according to the initial audio, each timbre feature and the first signal strength corresponding to each timbre feature need to be sent to other devices for the other devices to analyze and determine the timbre features that need to be collected subsequently. Specifically, it can be:

[0042] Based on ultra-wideband technology, each timbre feature and the first signal strength corresponding to each timbre feature are sent to each other device, so that each other device determines the target audio corresponding to each other device according to each timbre feature and the first signal strength corresponding to each timbre feature.

[0043] Among them, ultra-wideband technology (Ultra Wide Band, UWB) is used to achieve extremely low-latency communication between devices, ensuring that information exchange between devices is fast and reliable, and reducing the delay problem of audio synchronization.

[0044] Specifically, based on the ultra-wideband technology that can quickly broadcast and receive signals, the audio device can send the analyzed timbre characteristics and the first signal strength corresponding to each timbre characteristic to each other device, so that each other device can receive this information, and can determine the target audio corresponding to each other device based on each timbre characteristic and the first signal strength corresponding to each timbre characteristic, that is, each other device can know the first signal strength of each timbre characteristic of the audio device, so as to determine which timbre characteristics to collect specifically.

[0045] Exemplarily, the audio device and other devices can periodically broadcast and receive timbre features and signal strength information corresponding to each timbre feature. Each device can periodically (e.g., once per second, etc.) broadcast the timbre features and the first signal strength corresponding to each timbre feature it has collected, and can receive information (each timbre feature and the second signal strength corresponding to each timbre feature) broadcast by other devices, and store it in a local data structure, so as to facilitate comparative analysis and determine the target timbre feature.

[0046] S130, processing the initial audio according to the target timbre feature to obtain a target audio corresponding to the target timbre feature, and sending the target audio to other devices so that the other devices play the target audio with the same signal strength as the audio device.

[0047] The target audio is a portion of the initial audio that corresponds to the target timbre feature.

[0048] Specifically, the initial audio is screened and filtered according to the target timbre feature to obtain the target audio containing only the target timbre feature. Furthermore, the target audio can be sent to each other device so that each other device plays the target audio with the same signal strength as the audio device. It is understandable that the audio device does not play the target audio, but the audio device can receive the audio sent by each other device with the same signal strength as the other devices and play it.

[0049] Optionally, for the target audio, an audio enhancement algorithm may be used to improve the audio effect, ensuring that the signal strength of the target audio can be further enhanced so that other devices can play the target audio more clearly.

[0050] The audio processing method provided in this embodiment obtains the initial audio in real time, determines the timbre features included in the initial audio and the first signal strength corresponding to each timbre feature according to the initial audio, so as to separate the timbre features, and receives the timbre features sent by other devices and the second signal strength corresponding to each timbre feature. For each timbre feature, the target timbre feature is determined from each timbre feature according to the first signal strength and each second signal strength corresponding to the timbre feature, so as to determine the acquisition device corresponding to each timbre feature and ensure the optimal audio acquisition method. Further, the initial audio is processed according to the target timbre feature to obtain the target audio corresponding to the target timbre feature, so as to ensure that the audio device only acquires the target audio and avoid interference from other timbre signals. The target audio is sent to other devices so that each other device plays the target audio with the same signal strength as the audio device, thereby realizing audio collaborative processing of multiple devices, improving the robustness and stability of audio acquisition, processing and playback, and optimizing the resource allocation of audio processing through the strongest signal device responsibility system, dispersing the processing load, and improving the overall efficiency and audio clarity.

[0051] The above embodiment describes the audio processing method. In addition to the anti-interference processing of the initial audio, the emphasized sending of each timbre feature and the first signal corresponding to each timbre feature, in this embodiment, the analysis and determination of the timbre feature and the first signal strength, the determination of the target timbre feature, and the method of extracting the audio corresponding to the target timbre feature can be further described, such as Figure 2 shown. Figure 2This is a flowchart of another audio processing method provided by an embodiment of the present application, wherein the explanations of the terms that are the same or corresponding to the above-mentioned embodiments are not repeated here. Figure 2 As shown, the method may specifically include the following steps:

[0052] S210, acquiring the initial audio in real time, determining the Mel-cepstral coefficients corresponding to the initial audio according to the initial audio, and determining various timbre features included in the initial audio according to the Mel-cepstral coefficients.

[0053] Among them, Mel-scale Frequency Cepstral Coefficients (MFCC) are linear transformations of the logarithmic energy spectrum of the nonlinear Mel scale based on the sound frequency, and can be used for timbre analysis.

[0054] Specifically, the audio of all speakers in the current space is collected, and anti-interference processing can be performed to obtain the initial audio, and further, the initial audio is subjected to Mel frequency analysis and cepstrum analysis to obtain Mel cepstrum coefficients. By analyzing the Mel cepstrum coefficients, various timbre features contained in the initial audio can be determined.

[0055] S220: For each timbre feature, determine a portion of audio corresponding to the timbre feature from the initial audio, and calculate a first signal strength corresponding to the timbre feature based on the portion of audio.

[0056] The partial audio is a portion of the initial audio corresponding to each timbre feature, that is, a portion of the initial audio reflecting the timbre feature.

[0057] Specifically, the initial audio is split according to each timbre feature to obtain partial audio corresponding to each timbre feature, that is, each partial audio segment can correspond to a timbre feature. Further, for each timbre feature, the signal strength of the partial audio corresponding to the timbre feature is calculated to obtain a first signal strength corresponding to the timbre feature.

[0058] It is understandable that each device can collect and analyze the timbre characteristics of all speakers in the current space, and determine and judge the strength (first signal strength) of the speech signal (partial audio) corresponding to each timbre characteristic. Taking the audio device as an example, it can be specifically that after the audio device is started, the microphone array of the audio device starts to collect audio data and determine the initial audio. Then, the signal processing algorithm can be used to extract the timbre characteristics (such as MFCC characteristics, etc.) in the initial audio, which can ensure accurate recognition and separation of the voices of each speaker, and can calculate the signal strength corresponding to each timbre characteristic, that is, the first signal strength.

[0059] S230: Receive each timbre feature sent by each other device and a second signal strength corresponding to each timbre feature.

[0060] S240: For each timbre feature, in response to the first signal strength corresponding to the timbre feature being greater than each second signal strength, determining the timbre feature as a target timbre feature.

[0061] Specifically, for each timbre feature, if the first signal strength corresponding to the timbre feature in the audio device is greater than the second signal strength corresponding to each other device, it means that the audio device has the best collection effect for the timbre feature. Therefore, the timbre feature is determined as the target timbre feature, and the audio device can be used to collect the audio corresponding to the timbre feature separately subsequently.

[0062] It can be understood that, for each timbre feature, if the first signal strength corresponding to the timbre feature in the audio device is less than the second signal strength corresponding to the timbre feature in at least one other device, it means that the audio device's collection effect for the timbre feature is not optimal, and therefore the audio device is not required to collect the audio corresponding to the timbre feature in the future.

[0063] S250. For each timbre feature, in response to the first signal strength of the timbre feature being equal to at least one second signal strength, a candidate device corresponding to the timbre feature is determined; in response to the identifier of the audio device being the same as the identifier of the randomly selected target device, the timbre feature is determined to be a target timbre feature.

[0064] The devices to be selected are at least two devices having the same strongest signal strength in a certain timbre feature. The target device is any one of the devices to be selected, and can be randomly selected from the devices to be selected.

[0065] Specifically, for each timbre feature, if the first signal strength of the timbre feature is equal to at least one second signal strength, it means that although the audio device has the best acquisition effect for the timbre feature, there are devices with the same acquisition effect among other devices. Therefore, the audio device is used as the candidate device corresponding to the timbre feature. In addition, at least one candidate device can also be determined among other devices. The identifier of the target device can be randomly determined by the audio device, or the audio device can receive the identifier of the target device sent by other candidate devices. If the identifier of the audio device is the same as the identifier of the randomly selected target device, the timbre feature is determined to be the target timbre feature, and the audio device can be used to separately acquire the audio corresponding to the timbre feature later.

[0066] Optionally, the target device can be determined from the devices to be selected in the following manner: the devices can be sorted in advance according to their identifiers, and the device ranked first in the list of the devices to be selected can be randomly selected to determine the target device from the devices to be selected including the device itself, so that it can be known whether the device itself is the target device, and the identifier of the target device can be sent to the other devices to be selected. In this way, the other devices to be selected can determine whether they are the target device based on the received identifier of the target device.

[0067] It can be understood that, from a macro level, it is possible to determine on which device the collection effect of each timbre feature is strongest. Specifically, in each device, the signal strengths of the timbre features of all devices stored locally are compared to determine on which device each timbre feature is strongest, and the results are recorded to facilitate subsequent audio collection and extraction.

[0068] S260: Input the target timbre feature and the initial audio into a pre-built timbre separation model, and extract the target audio corresponding to the target timbre feature from the initial audio.

[0069] Among them, the timbre separation model is a pre-built model used for timbre separation and filtering, which can remove other timbres except the target timbre characteristics.

[0070] Specifically, the target timbre features and the initial audio are input into a pre-built timbre separation model together. The timbre separation model can be used to separate the target timbre features from the part corresponding to the initial audio as the target audio, so as to extract the target audio corresponding to the target timbre features from the initial audio, so as to facilitate subsequent sending to other devices.

[0071] It is understandable that the device with the strongest signal strength is responsible for collecting the audio with the corresponding timbre characteristics and filtering out the audio with other timbre characteristics. For audio devices, after determining that it is the target device corresponding to a certain timbre characteristic, it can start to specifically collect the sound of the timbre characteristic, that is, the target audio. Specifically, it can be to use a timbre separation model (such as a deep learning model) to separate and filter the timbre and remove other timbre from the current audio. Due to the use of advanced algorithms such as deep learning models, filtering and separation of specific timbre characteristics can be achieved, ensuring that the target device can clearly capture the voice of the target speaker.

[0072] S270: Send the target audio to each other device, so that each other device plays the target audio with the same signal strength as the audio device.

[0073] The hardware architecture of the audio device needs to be configured with a UWB module to achieve low-latency communication between devices. It also needs to be configured with a microphone array and speakers to collect and play the sound in the current space. It also needs to be configured with a DSP (Digital Signal Processing) or SoC (System on Chip) to process audio.

[0074] Through the above method, the device with the strongest signal can be realized, that is, each timbre feature is collected and processed by the strongest device, which disperses the load of audio processing and optimizes system performance. In addition, multi-device collaboration is realized, specifically, multiple devices cooperate with each other through UWB communication to jointly judge and process audio signals, avoiding single point failure and improving system reliability.

[0075] Exemplarily, each device can utilize the UWB broadcast mechanism to maintain extremely low latency to interact with nearby devices, which collect and play audio in the same space. Each device opens the microphone to collect the timbre characteristics of all speakers within the range, determines that there are N people, and calculates the signal strength of each person's speech. For example: the devices are S1, S2, S3 and S4, and the speakers are R1, R2, R3 and R4. Then, each device interacts with nearby devices to determine which devices have the strongest signal for collecting each person's timbre characteristics. The device with the strongest signal is responsible for collecting the audio of the timbre characteristics, and the model algorithm can be used to filter out the parts of other timbre characteristics. For example: the timbre characteristics of speaker R4 have the strongest signal on device S2, then S2 will not play R4's audio, but will only collect and send it out. However, S2 will play the audio of R1, R2 and R3, and because the device itself has an echo cancellation function, it will not collect and play the sound played by the device's own speaker again. However, S2 will collect the sounds of R1, R2, R3 and R4 played by other devices (S1, S3 and S4), so they need to be removed using phase delay algorithm and audio cancellation algorithm.

[0076] The above-mentioned technical solutions are highly adaptable and can be applied to scenarios of different scales. They are highly flexible and scalable, and can be applied to different scales from small meeting rooms to large conference halls. Moreover, they are not only applicable to conference scenarios, but can also be extended to other scenarios that require audio processing and communication, such as distance education, collaborative office, etc.

[0077] The audio processing method provided in this embodiment obtains the initial audio in real time, determines the Mel-frequency cepstral coefficients corresponding to the initial audio according to the initial audio, determines each timbre feature according to the Mel-frequency cepstral coefficients, determines the partial audio corresponding to the timbre feature from the initial audio for each timbre feature, and calculates the first signal strength corresponding to the timbre feature according to the partial audio, determines the timbre feature as the target timbre feature in response to the first signal strength corresponding to the timbre feature being greater than each second signal strength, determines the timbre feature as the target timbre feature for each timbre feature, determines the device to be selected in response to the first signal strength of the timbre feature being equal to at least one second signal strength for the timbre feature, and determines the timbre feature as the target timbre feature in response to the identification of the audio device being the same as the identification of the randomly selected target device, thereby optimizing the resource allocation of audio processing through the strongest signal device responsibility system, avoiding unnecessary waste of resources, and being able to perform multi-device processing. Working together, the processing load is dispersed, and the overall audio processing efficiency is improved. Then, the target timbre characteristics and the initial audio are input into a pre-built timbre separation model, and the target audio corresponding to the target timbre characteristics is extracted from the initial audio, so as to ensure that each speaker's voice is clearly audible and reduce background noise and interference through intelligent timbre recognition and separation. Furthermore, the target audio is sent to other devices so that each other device plays the target audio. Low-latency UWB communication can be used to ensure audio synchronization between devices, avoid delays and misalignment problems of audio signals, and improve audio quality. Since echo cancellation algorithms, phase delay algorithms, and audio cancellation algorithms are also applied, echo and crosstalk problems can be effectively reduced, providing a more natural and smooth conversation experience. The use of a distributed audio processing mechanism can enable multiple devices to work together to improve the robustness and stability of overall audio acquisition, processing, and playback.

[0078] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0079] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0080] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides an audio processing device. Figure 3 A schematic diagram of the structure of an audio processing device provided in an embodiment of the present application, referring to Figure 3 The audio processing device is configured in an audio device with audio function, and includes: a collection and processing module 310, a target timbre feature determination module 320 and an audio processing module 330.

[0081] Among them, the acquisition and processing module 310 is used to obtain the initial audio in real time, and determine the timbre features included in the initial audio and the first signal strength corresponding to each timbre feature based on the initial audio; the target timbre feature determination module 320 is used to receive the timbre features sent by each other device and the second signal strength corresponding to each timbre feature, and for each timbre feature, determine the target timbre feature from each timbre feature based on the first signal strength corresponding to the timbre feature and the second signal strengths; the audio processing module 330 is used to process the initial audio according to the target timbre feature, obtain the target audio corresponding to the target timbre feature, and send the target audio to each other device so that each other device plays the target audio with the same signal strength as the audio device.

[0082] Based on the above examples, optionally, the target timbre feature determination module 320 is also used to determine that the timbre feature is a target timbre feature in response to the first signal strength corresponding to the timbre feature being greater than each second signal strength; to determine the device to be selected corresponding to the timbre feature in response to the first signal strength of the timbre feature being equal to at least one second signal strength; and to determine that the timbre feature is a target timbre feature in response to the identifier of the audio device being the same as the identifier of the randomly selected target device.

[0083] Based on the above example, optionally, the acquisition and processing module 310 is also used to determine the Mel-cepstral coefficients corresponding to the initial audio based on the initial audio; determine the timbre features included in the initial audio based on the Mel-cepstral coefficients; for each timbre feature, determine the partial audio corresponding to the timbre feature from the initial audio, and calculate the first signal strength corresponding to the timbre feature based on the partial audio.

[0084] Based on the above example, optionally, the audio processing module 330 is also used to input the target timbre feature and the initial audio into a pre-built timbre separation model, and extract the target audio corresponding to the target timbre feature from the initial audio.

[0085] Based on the above example, optionally, after determining each timbre feature and the first signal strength corresponding to each timbre feature according to the initial audio, it also includes: an audio transmission module, which is used to send each timbre feature and the first signal strength corresponding to each timbre feature to each other device based on ultra-wideband technology, so that each other device determines the target audio corresponding to each other device according to each timbre feature and the first signal strength corresponding to each timbre feature.

[0086] Based on the above example, optionally, the acquisition processing module 310 is further used to acquire the current audio in real time, and based on the echo cancellation algorithm, remove the audio played by the audio device from the current audio to obtain the initial audio.

[0087] Based on the above example, optionally, the acquisition processing module 310 is also used to acquire the current audio in real time, and based on the phase delay algorithm and the audio cancellation algorithm, remove the audio played by each other device from the current audio to obtain the initial audio.

[0088] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0089] The device of the above embodiment is used to implement the corresponding audio processing method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0090] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the audio processing method described in any of the above embodiments is implemented.

[0091] Figure 4 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0092] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0093] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0094] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0095] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0096] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0097] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0098] The electronic device of the above embodiment is used to implement the corresponding audio processing method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0099] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the audio processing method described in any of the above embodiments.

[0100] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0101] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the audio processing method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0102] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0103] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0104] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0105] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. An audio processing method, characterized in that: Applicable to audio devices with audio functions, including: Acquire initial audio in real time, and determine, based on the initial audio, various timbre features included in the initial audio and first signal strengths corresponding to the various timbre features; Receiving each timbre feature and the second signal strength corresponding to each timbre feature sent by each other device, and for each timbre feature, determining a target timbre feature from each timbre feature according to the first signal strength corresponding to the timbre feature and each second signal strength; The initial audio is processed according to the target timbre feature to obtain a target audio corresponding to the target timbre feature, and the target audio is sent to each other device so that each other device plays the target audio with the same signal strength as the audio device.

2. The method according to claim 1, characterized in that: The step of determining a target timbre feature from each timbre feature according to the first signal strength and each second signal strength corresponding to the timbre feature comprises: In response to the first signal strength corresponding to the timbre feature being greater than each second signal strength, determining the timbre feature to be a target timbre feature; In response to the first signal strength of the timbre feature being equal to at least one second signal strength, the device to be selected corresponding to the timbre feature is determined; in response to the identifier of the audio device being the same as the identifier of the randomly selected target device, the timbre feature is determined to be the target timbre feature.

3. The method according to claim 1, characterized in that The step of determining, according to the initial audio, each timbre feature included in the initial audio and a first signal strength corresponding to each timbre feature comprises: Determine, according to the initial audio, a Mel-frequency cepstral coefficient corresponding to the initial audio; Determining various timbre features included in the initial audio according to the Mel-frequency cepstral coefficients; For each timbre feature, a portion of audio corresponding to the timbre feature is determined from the initial audio, and a first signal strength corresponding to the timbre feature is calculated based on the portion of audio.

4. The method according to claim 1, characterized in that: The processing of the initial audio according to the target timbre feature to obtain the target audio corresponding to the target timbre feature includes: The target timbre feature and the initial audio are input into a pre-built timbre separation model, and the target audio corresponding to the target timbre feature is extracted from the initial audio.

5. The method according to claim 1, characterized in that After determining each timbre feature and the first signal strength corresponding to each timbre feature according to the initial audio, the method further includes: Based on ultra-wideband technology, each timbre feature and the first signal strength corresponding to each timbre feature are sent to each other device, so that each other device determines the target audio corresponding to each other device according to each timbre feature and the first signal strength corresponding to each timbre feature.

6. The method according to claim 1, characterized in that The real-time acquisition of initial audio includes: The current audio is collected in real time, and based on an echo cancellation algorithm, the audio played by the audio device is removed from the current audio to obtain the initial audio.

7. The method according to claim 1, characterized in that The real-time acquisition of initial audio includes: The current audio is collected in real time, and based on a phase delay algorithm and an audio cancellation algorithm, the audio played by each other device is removed from the current audio to obtain the initial audio.

8. An audio processing device, characterized in that: Configured for audio devices with audio functions, including: The acquisition and processing module is used to acquire the initial audio in real time, and determine, according to the initial audio, each timbre feature included in the initial audio and the first signal strength corresponding to each timbre feature; A target timbre feature determination module is used to receive each timbre feature and the second signal strength corresponding to each timbre feature sent by each other device, and for each timbre feature, determine the target timbre feature from each timbre feature according to the first signal strength and each second signal strength corresponding to the timbre feature; The audio processing module is used to process the initial audio according to the target timbre feature to obtain the target audio corresponding to the target timbre feature, and send the target audio to each other device so that each other device plays the target audio with the same signal strength as the audio device.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the audio processing method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the audio processing method according to any one of claims 1 to 7.

Images