Audio data processing method, apparatus, system, electronic device, and storage medium

By acquiring multi-channel audio data and processing it according to the audio usage scenario, the problem that existing technologies cannot meet the requirements for high-quality audio in multiple scenarios is solved, and high-quality audio processing is achieved in different scenarios.

CN117389506BActive Publication Date: 2026-11-10IFLYTEK CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202311412427.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-10-26
Publication Date
2026-11-10
Estimated Expiration
2043-10-26

AI Technical Summary

Technical Problem

Existing technologies cannot meet users' high-quality audio needs in various usage scenarios, nor can they adapt to the needs of different audio usage scenarios.

Method used

By acquiring multi-channel audio data, the number of audio channels required by the target application is determined, and the audio data is processed according to the audio usage scenario. This includes processing the data using specific algorithms when the number of audio channels is less than the preset number, and directly sending the data to the target application for processing when the number of audio channels is not less than the preset number.

Benefits of technology

It enables the acquisition of audio that meets the needs of target applications in different audio usage scenarios, improves the quality of audio usage, and meets users' high-quality audio needs in multiple scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117389506B_ABST
    Figure CN117389506B_ABST
Patent Text Reader

Abstract

The application provides an audio data processing method and device, electronic equipment and storage medium. The audio data processing method comprises: acquiring multi-channel audio data and determining the number of audio channels of audio data required by a target application; in the case that the number of audio channels is less than a preset channel number, processing the multi-channel audio data according to an audio use scenario of the target application to obtain first target audio conforming to the audio use scenario, and sending the first target audio to the target application; in the case that the number of audio channels is not less than the preset channel number, sending the multi-channel audio data to the target application to enable the target application to process the multi-channel audio data to obtain second target audio conforming to the audio use scenario of the target application.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of audio processing, specifically to an audio data processing method, apparatus, system, electronic device, and storage medium. Background Technology

[0002] Recording is a basic function of devices with audio signal modules. Multi-channel recording based on a recording matrix composed of multiple microphones is also a common method to improve recording quality.

[0003] To improve recording quality, existing technologies typically optimize audio based on the specific usage scenario of the recorded audio, thereby providing audio with the quality required for that scenario. However, this approach is only applicable to a single usage scenario and cannot cover multiple usage scenarios of audio, thus failing to meet users' pursuit of high-quality recordings across multiple scenarios.

[0004] Therefore, how to process audio data to meet users' needs for high-quality audio in multiple scenarios has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] Based on the aforementioned technological status, this application proposes an audio data processing method, apparatus, system, electronic device, and storage medium to meet users' needs for high-quality audio in various scenarios.

[0006] To achieve the above-mentioned technical objectives, this application proposes the following technical solution:

[0007] An audio data processing method, comprising:

[0008] Acquire multi-channel audio data and determine the number of audio channels required by the target application.

[0009] If the number of audio channels is less than the preset number of channels, the multi-channel audio data is processed according to the audio usage scenario of the target application to obtain a first target audio that conforms to the audio usage scenario, and the first target audio is sent to the target application;

[0010] If the number of audio channels is not less than the preset number of channels, the multi-channel audio data is sent to the target application so that the target application processes the multi-channel audio data to obtain a second target audio that conforms to the audio usage scenario of the target application.

[0011] Optionally, the audio usage scenario of the target application is determined in the following way:

[0012] Obtain the audio usage conditions of the target application, which include the number of audio channels required by the target application for audio data, and the recording source of the audio required by the target application.

[0013] Based on the audio usage conditions of the target application, determine the audio usage scenario of the target application.

[0014] Optionally, processing the multi-channel audio data to obtain a first target audio that conforms to the audio usage scenario of the target application includes:

[0015] Based on the audio usage scenario, determine the audio processing algorithm corresponding to the audio usage scenario;

[0016] Using an audio processing algorithm corresponding to the audio usage scenario, the multi-channel audio data is processed to obtain a first target audio that conforms to the audio usage scenario.

[0017] Optionally, determining the audio processing algorithm corresponding to the audio usage scenario includes:

[0018] When the audio usage scenario is audio recording, the audio processing algorithm is determined to be an audio noise reduction algorithm and an audio enhancement algorithm;

[0019] as well as,

[0020] When the audio usage scenario is call recording, the audio processing algorithm is determined to be an echo cancellation algorithm.

[0021] Optionally, the target application processes the multi-channel audio data in the following manner to obtain a second target audio that conforms to the audio usage scenario of the target application:

[0022] The audio processing algorithm corresponding to the audio usage scenario of the target application is invoked to process the multi-channel audio data and obtain a second target audio that conforms to the audio usage scenario of the target application.

[0023] or,

[0024] From the audio processing algorithms stored in the target application, an algorithm corresponding to the audio usage scenario of the target application is selected, and the multi-channel audio data is processed to obtain a second target audio that conforms to the audio usage scenario of the target application.

[0025] Optionally, the use cases for the second target audio include at least one of voice wake-up recognition and voice evaluation.

[0026] An audio data processing apparatus, comprising:

[0027] The first unit is used to acquire multi-channel audio data and determine the number of audio channels required by the target application.

[0028] The second unit is used to process the multi-channel audio data to obtain a first target audio that conforms to the audio usage scenario of the target application when the number of audio channels is less than the preset number of channels, and to send the first target audio to the target application.

[0029] The third unit is used to send the multi-channel audio data to the target application when the number of audio channels is not less than the preset number of channels, so that the target application can process the multi-channel audio data to obtain a second target audio that conforms to the audio usage scenario of the target application.

[0030] An audio data processing system, comprising:

[0031] Audio acquisition unit, used to acquire multi-channel audio data;

[0032] An audio processing unit is configured to acquire multi-channel audio data and determine the number of audio channels required by a target application; if the number of audio channels is less than a preset number of channels, the unit processes the multi-channel audio data according to the audio usage scenario of the target application to obtain a first target audio that conforms to the audio usage scenario, and sends the first target audio to the target application; if the number of audio channels is not less than the preset number of channels, the unit sends the multi-channel audio data to the target application so that the target application processes the multi-channel audio data to obtain a second target audio that conforms to the audio usage scenario of the target application.

[0033] An electronic device, comprising:

[0034] Processor and memory;

[0035] The memory is used to store computer programs; the processor is used to implement the above-mentioned audio data processing method by running the computer programs stored in the memory.

[0036] A computer storage medium storing a computer program that, when executed, implements the above-described audio data processing method.

[0037] This application provides an audio data processing method, apparatus, system, electronic device, and storage medium. The audio data processing method includes: acquiring multi-channel audio data and determining the number of audio channels required by a target application; if the number of audio channels is less than a preset number of channels, processing the multi-channel audio data according to the audio usage scenario of the target application to obtain a first target audio that conforms to the audio usage scenario, and sending the first target audio to the target application; if the number of audio channels is not less than the preset number of channels, sending the multi-channel audio data to the target application so that the target application processes the multi-channel audio data to obtain a second target audio that conforms to the audio usage scenario of the target application.

[0038] The audio data processing method processes multi-channel audio data in conjunction with the audio usage scenarios of the target application to obtain target audio that meets the needs of the target application, thereby satisfying users' audio usage needs in different scenarios and improving the quality of audio usage. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0040] Figure 1 This is a flowchart of an audio data processing method provided in an embodiment of this application;

[0041] Figure 2 This is a flowchart illustrating a first type of multi-channel audio data processing in an embodiment of this application.

[0042] Figure 3 This is a flowchart illustrating a second type of multi-channel audio data processing provided in an embodiment of this application.

[0043] Figure 4 This is a flowchart illustrating the processing of multi-channel audio data by the recording system provided in this embodiment of the application.

[0044] Figure 5 This is a structural diagram of the audio data processing apparatus provided in the embodiments of this application;

[0045] Figure 6 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0046] Recording is a basic function of devices with audio signal modules. Multi-channel recording based on a recording matrix composed of multiple microphones is also a common method to improve recording quality.

[0047] To improve recording quality, existing technologies typically optimize audio based on the specific usage scenario of the recorded audio, thereby providing audio with the quality required for that scenario. However, this approach is only applicable to a single usage scenario and cannot cover multiple usage scenarios of audio, thus failing to meet users' pursuit of high-quality recordings across multiple scenarios.

[0048] Therefore, this application provides an audio data processing method, apparatus, system, electronic device, and storage medium to meet users' needs for high-quality audio in multiple scenarios.

[0049] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0050] Exemplary methods

[0051] This application first proposes an audio data processing method, the core of which is to process multi-channel audio data in combination with the audio usage scenario of the target application, thereby obtaining target audio that meets the needs of the target application, satisfying the user's audio usage needs in different scenarios, and improving the audio usage quality.

[0052] In the embodiments of this application, the subject implementing the audio data processing method can be various types of user terminals or smart devices such as laptops, tablets, desktop computers, mobile devices (e.g., learning machines, mobile phones, host computers, smart robots, portable music players), or any combination of two or more of these data processing devices.

[0053] See Figure 1 The method includes:

[0054] S101. Acquire multi-channel audio data and determine the number of audio channels required by the target application.

[0055] The multi-channel audio data refers to audio signals that simultaneously contain multiple independent audio channels during recording or audio data processing, with each audio channel containing independent audio information.

[0056] The target application can be understood as an application. In one optional embodiment of this application, the target application refers to an application installed in the recording system. The application installed in the recording body can be one or more, wherein each application corresponds to a recording scenario. For example, the recording scenario can be an audio recording scenario, a call recording scenario, etc.

[0057] In another optional embodiment of this application, the target application and the recording system can also be connected via a data cable, wireless network, or other communication methods. For example, a mobile phone and a tablet computer can be connected via a super terminal. The target application can be an application installed on the mobile phone, such as an audio recording program or a voice wake-up program. The tablet computer is used to collect audio information.

[0058] Furthermore, the number of audio channels required by the target application can be sent by the target application to the recording system used for audio acquisition, so that the Audio Hardware Abstraction Layer (Audio HAL) in the recording body can know the number of audio channels required by the target application in a timely manner, thereby facilitating data processing of the acquired multi-channel audio to obtain audio that meets the required number of audio channels, and thus improving the audio quality obtained by the target application.

[0059] S102. If the number of audio channels is less than the preset number of channels, the multi-channel audio data is processed according to the audio usage scenario of the target application to obtain a first target audio that conforms to the audio usage scenario, and the first target audio is sent to the target application.

[0060] In one optional embodiment of this application, the target application can be divided into two categories based on the number of channels required for recording: single-channel or dual-channel recording and multi-channel recording. Single-channel or dual-channel recording includes audio usage scenarios such as audio recording, voice call recording, and camera recording; multi-channel recording includes audio usage scenarios such as voice wake-up and voice evaluation. The preset number of channels can be set to 3 based on the recording category. That is, if the number of audio channels is less than 3, the recording is single-channel or dual-channel; otherwise, the recording is multi-channel.

[0061] It is understood that setting the preset number of channels to 3 is only one optional implementation of this application. In actual application, the preset number of channels can be set according to the actual application scenario of the audio data processing method, or it can be set by the user according to actual needs, or it can be the number of audio channels automatically detected by the audio processing system according to different scenarios and tasks. This application does not limit this. For example, in some application scenarios, where the recording only includes single-channel recording and multi-channel recording, the preset number of channels can be 2.

[0062] In one optional embodiment of this application, the audio usage scenario of the target application is determined in the following way:

[0063] Obtain the audio usage conditions of the target application, which include the number of audio channels required by the target application and the recording source required by the target application; determine the audio usage scenario of the target application based on the audio usage conditions of the target application.

[0064] The recording source can be understood as the source of the audio signal used in the process of acquiring multi-channel audio data, indicating where the recording system obtains the audio input. For example, assuming that the target application requires a single or dual audio channel, and the recording source is a microphone, then the audio usage scenario of the target application can be determined to be ordinary audio recording.

[0065] Specifically, the audio usage scenario of the target application can be sent to the recording system after the user enables the recording function of the target application, so that the recording system can determine the audio usage scenario of the target application.

[0066] In another optional embodiment of this application, the target application may also directly send the corresponding audio usage scenario tag to the recording system so that the recording system can directly obtain the audio usage scenario of the target application.

[0067] When the number of audio channels is less than the preset number of channels, it indicates that the target application requires either a single channel or dual channels for recording. In this case, processing the multi-channel audio data according to the audio usage scenario of the target application to obtain a first target audio that conforms to the audio usage scenario includes:

[0068] Based on the audio usage scenario, an audio processing algorithm corresponding to the audio usage scenario is determined; using the audio processing algorithm corresponding to the audio usage scenario, the multi-channel audio data is processed to obtain a first target audio that conforms to the audio usage scenario.

[0069] For example, when the audio usage scenario is audio recording, the audio processing algorithm is specifically an audio noise reduction algorithm and an audio enhancement algorithm; when the audio usage scenario is call recording, the audio processing algorithm is determined to be an echo cancellation algorithm.

[0070] Furthermore, to facilitate understanding of the process of processing the multi-channel audio data when the number of audio channels is less than the preset number of channels, the following is combined with... Figure 2 The process will be described in detail.

[0071] like Figure 2 As shown, Figure 2 Includes:

[0072] Multi-channel raw audio 201, audio hardware abstraction layer 202, target application 203.

[0073] First, after acquiring the multi-channel raw audio through the hardware device of the recording system, the multi-channel raw audio 201 is sent to the audio hardware abstraction layer 202.

[0074] The Audio Hardware Abstraction Layer 202 is a software component used to manage audio hardware and provide audio functions, acting as an intermediary layer between the audio hardware and the operating system. The Audio Hardware Abstraction Layer 202 is also used to receive the number of audio channels in the audio data required by the target application, as well as the recording source of the audio required by the target application; and to determine the audio usage scenario of the target application based on the number of audio channels in the audio data required by the target application and the recording source of the audio required by the target application.

[0075] Furthermore, after determining the audio usage scenario of the target application, the audio hardware abstraction layer 202 calls the algorithm corresponding to the audio usage scenario to process the multi-channel audio data, obtains a first target audio that meets the number of audio channels required by the target application, and sends the first target audio to the target application 203.

[0076] For example, assuming the original multi-channel audio consists of 8 audio signals and the audio usage scenario of the target application is a call scenario, the algorithm corresponding to the call scenario is determined to be an echo cancellation algorithm. In this case, the third output of the algorithm is read. This output mainly performs echo cancellation and other algorithm processing on the multi-channel audio to convert the 8 audio signals into 1 audio signal for output to the target application.

[0077] For example, assuming the multi-channel original audio consists of 8 audio signals, and the audio usage scenario of the target application is an audio recording scenario, the algorithm corresponding to the call scenario is determined to be a noise reduction and enhancement algorithm. In this case, the fourth output of the algorithm is read. This output mainly performs noise reduction and enhancement processing to convert the 8 audio signals into 1 audio signal for output to the target application.

[0078] S103. If the number of audio channels is not less than the preset number of channels, the multi-channel audio data is sent to the target application so that the target application processes the multi-channel audio data to obtain a second target audio that conforms to the audio usage scenario of the target application.

[0079] For example, when the number of audio channels is not less than 3, it means that the target application requires multiple audio channels. In the prior art, considering that the native Android system logic only supports single-channel or dual-channel recording and does not support multi-channel recording, but applications such as wake-up recognition and voice evaluation often need multiple raw audio channels to improve recognition efficiency. In the prior art, usually after the audio data reaches the audio hardware abstraction layer, one or two audio channels are directly sent to the relevant application, or after some algorithm processing, the multi-channel audio data is converted into one or two audio channels and sent to the relevant application. However, neither of these methods can meet the high-quality audio requirements of wake-up recognition, voice evaluation and other scenarios.

[0080] In this embodiment, the recording system can be made to support multi-channel recording by modifying the software logic. If the number of audio channels required by the target application is greater than or equal to 3, it indicates that the audio usage scenario of the target application may be wake-up recognition, voice evaluation, etc. In this case, the multi-channel audio data can be directly sent to the target application based on the above step S103, so that the target application can process the multi-channel audio data to obtain a second target audio that conforms to the audio usage scenario of the target application.

[0081] Specifically, the target application can process the multi-channel audio data in the following ways to obtain a second target audio that matches the audio usage scenario of the target application:

[0082] The audio processing algorithm corresponding to the audio usage scenario of the target application is invoked to process the multi-channel audio data and obtain a second target audio that conforms to the audio usage scenario of the target application.

[0083] or,

[0084] From the audio processing algorithms stored in the target application, an algorithm corresponding to the audio usage scenario of the target application is selected, and the multi-channel audio data is processed to obtain a second target audio that conforms to the audio usage scenario of the target application.

[0085] That is, the target application can retrieve the corresponding algorithm from the audio hardware abstraction layer of the recording system according to its own audio usage scenario, process the multi-channel audio data, and then obtain the second target audio; or it can write the corresponding algorithm at the bottom layer of the target application, so that after receiving the multi-channel audio, the target application can directly use the algorithm to process the multi-channel audio and obtain the second target audio.

[0086] Furthermore, to facilitate understanding of the process of processing the multi-channel audio data when the number of audio channels is not less than the preset number of channels, the following is combined with... Figure 3 The process will be described in detail.

[0087] like Figure 3 As shown, Figure 3 Includes:

[0088] Multi-channel raw audio 301, audio hardware abstraction layer 302, target application 303, algorithm processing module 304.

[0089] After the recording system acquires the multi-channel raw audio 301, it sends the multi-channel raw audio 301 to the audio hardware abstraction layer 302. Then, if the number of audio channels required by the target application received by the audio hardware abstraction layer 302 is not less than the preset number of channels, the multi-channel raw audio 301 is directly sent to the corresponding target application.

[0090] After obtaining the multi-channel original audio 301, the target application 303 can combine the audio usage scenario of the application and call the algorithm in the audio hardware abstraction layer 302 to obtain the second target audio, so as to improve the recording quality.

[0091] Alternatively, if a corresponding algorithm is deployed in the target application 303, after the target application 303 obtains the multi-channel original audio, the algorithm can be used to process the multi-channel original audio 301 to obtain the second target audio, thereby improving the recording quality.

[0092] Furthermore, to facilitate understanding of the audio data processing method provided in the embodiments of this application, the following is combined with... Figure 4 The audio data processing method will be described in detail below.

[0093] like Figure 4 As shown, Figure 4It includes: an audio digital signal processor 401, a kernel 402, an audio hardware abstraction layer 403, an audio framework 404, and an application program 405.

[0094] Applications 405 include: voice assessment applications, wake-up recognition applications, call applications, and audio recording applications.

[0095] Taking a preset number of channels of 3 as an example, the number of audio channels required by the call application and the audio recording application is less than 3 (that is, the number of audio channels required by such applications is single-channel or dual-channel); the number of audio channels required by the voice evaluation application and the wake-up recognition application is greater than 3 (that is, the number of audio channels required by such applications is multi-channel).

[0096] For the aforementioned call-related applications and audio recording applications, taking call-related applications as an example, when the call-related application enables the recording function, the call-related application sends the number of audio channels required by the application and the audio recording source required to the recording system so that the recording system can determine the audio usage scenario of the voice call application based on the number of voice channels required by the call-related application and the audio recording source required.

[0097] Furthermore, the recording system collects multi-channel audio data from the recording source and sends it to the audio hardware abstraction layer 403 via the audio digital signal processor 401 and the kernel 402. The audio hardware abstraction layer 403 deploys multiple audio processing algorithms. After receiving the multi-channel audio, the audio hardware abstraction layer 403 determines the echo cancellation algorithm and other algorithms corresponding to the audio usage scenario of the call application, and uses the echo cancellation algorithm and other algorithms to process the multi-channel audio data to obtain single-channel audio or multi-channel audio. The single-channel audio or multi-channel audio is then sent to the call application via the audio framework 404.

[0098] For the aforementioned voice evaluation applications and wake-up recognition applications, taking the voice evaluation application as an example, when the voice evaluation application starts the recording function, the voice evaluation application sends the number of audio channels it needs to the recording system. After determining that the number of audio channels needed by the voice evaluation application is greater than 3, the recording system directly sends the collected multi-channel audio data to the voice evaluation application.

[0099] After receiving the multi-channel audio data, the voice evaluation application can combine the audio usage scenario of the voice evaluation application and call the voice evaluation algorithm corresponding to the audio usage scenario from the audio hardware abstraction layer 403 to process the multi-channel audio data to obtain single-channel or dual-channel audio that conforms to the audio usage scenario.

[0100] Alternatively, if the speech evaluation application has a speech evaluation algorithm corresponding to the audio usage scenario, after receiving the multi-channel audio data, the speech evaluation application can directly use the speech evaluation algorithm to process the multi-channel speech to obtain single-channel or dual-channel audio that meets the audio usage scenario.

[0101] In summary, the audio data processing method described above processes multi-channel audio data in conjunction with the audio usage scenarios of the target application, thereby obtaining target audio that meets the needs of the target application, satisfying users' audio usage needs in different scenarios, and improving the quality of audio usage.

[0102] Exemplary device

[0103] Corresponding to the above-described audio data processing method, this application also provides an audio data processing apparatus, see below. Figure 5 The device includes:

[0104] Unit 501 is used to acquire multi-channel audio data and determine the number of audio channels required by the target application.

[0105] The second unit 502 is used to process the multi-channel audio data according to the audio usage scenario of the target application when the number of audio channels is less than the preset number of channels, to obtain a first target audio that conforms to the audio usage scenario, and to send the first target audio to the target application.

[0106] The third unit 503 is used to send the multi-channel audio data to the target application when the number of audio channels is not less than the preset number of channels, so that the target application can process the multi-channel audio data to obtain a second target audio that conforms to the audio usage scenario of the target application.

[0107] As an optional implementation, the audio usage scenario of the target application is determined in the following way:

[0108] Obtain the audio usage conditions of the target application, which include the number of audio channels required by the target application for audio data, and the recording source of the audio required by the target application.

[0109] Based on the audio usage conditions of the target application, determine the audio usage scenario of the target application.

[0110] As an optional implementation, processing the multi-channel audio data to obtain a first target audio that conforms to the audio usage scenario of the target application includes:

[0111] Based on the audio usage scenario, determine the audio processing algorithm corresponding to the audio usage scenario;

[0112] Using an audio processing algorithm corresponding to the audio usage scenario, the multi-channel audio data is processed to obtain a first target audio that conforms to the audio usage scenario.

[0113] As an optional implementation, determining the audio processing algorithm corresponding to the audio usage scenario includes:

[0114] When the audio usage scenario is audio recording, the audio processing algorithm is determined to be an audio noise reduction algorithm and an audio enhancement algorithm;

[0115] as well as,

[0116] When the audio usage scenario is call recording, the audio processing algorithm is determined to be an echo cancellation algorithm.

[0117] As an optional implementation, the target application processes the multi-channel audio data in the following manner to obtain a second target audio that conforms to the audio usage scenario of the target application:

[0118] The audio processing algorithm corresponding to the audio usage scenario of the target application is invoked to process the multi-channel audio data and obtain a second target audio that conforms to the audio usage scenario of the target application.

[0119] or,

[0120] From the audio processing algorithms stored in the target application, an algorithm corresponding to the audio usage scenario of the target application is selected, and the multi-channel audio data is processed to obtain a second target audio that conforms to the audio usage scenario of the target application.

[0121] As an optional implementation, the use cases of the second target audio include at least one of voice wake-up recognition and voice evaluation.

[0122] The audio data processing apparatus provided in this embodiment belongs to the same concept as the audio data processing method provided in the above embodiments of this application. It can execute the audio data processing method provided in any of the above embodiments of this application and has the corresponding functional modules and beneficial effects for executing the audio data processing method. Technical details not described in detail in this embodiment can be found in the specific processing content of the audio data processing method provided in the above embodiments of this application, and will not be repeated here.

[0123] The functions implemented by the first unit 501, the second unit 502, and the third unit 503 can be implemented by the same or different processors, and this application embodiment does not limit this.

[0124] It should be understood that the units in the above device can be implemented by a processor calling software. For example, the device includes a processor connected to a memory containing instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of each unit in the device. The processor can be a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal or external to the device. Alternatively, the units in the device can be implemented as hardware circuits. By designing the hardware circuits, some or all of the unit functions can be implemented. The hardware circuits can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are implemented by designing the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a PLD, such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through configuration files to implement the functions of some or all of the above units. All units in the above device can be implemented entirely by a processor calling software, entirely by hardware circuits, or partially by a processor calling software with the remaining parts implemented by hardware circuits.

[0125] In this application embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU, or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0126] As can be seen, each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0127] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a System-on-Chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.

[0128] Exemplary System

[0129] Another embodiment of this application also proposes an audio data processing system, the system comprising:

[0130] Audio acquisition unit, used to acquire multi-channel audio data;

[0131] An audio processing unit is configured to acquire multi-channel audio data and determine the number of audio channels required by a target application; if the number of audio channels is less than a preset number of channels, the unit processes the multi-channel audio data according to the audio usage scenario of the target application to obtain a first target audio that conforms to the audio usage scenario, and sends the first target audio to the target application; if the number of audio channels is not less than the preset number of channels, the unit sends the multi-channel audio data to the target application so that the target application processes the multi-channel audio data to obtain a second target audio that conforms to the audio usage scenario of the target application.

[0132] The audio data processing system provided in this embodiment belongs to the same concept as the audio data processing method provided in the above embodiments of this application. It can execute the audio data processing method provided in any of the above embodiments of this application and has the corresponding functional modules and beneficial effects for executing the audio data processing method. Technical details not described in detail in this embodiment can be found in the specific processing content of the audio data processing method provided in the above embodiments of this application, and will not be repeated here.

[0133] Exemplary electronic devices

[0134] Another embodiment of this application also provides an electronic device, see [link to relevant documentation] Figure 6 As shown, the electronic device includes:

[0135] Memory 200 and processor 210;

[0136] The memory 200 is connected to the processor 210 and is used to store programs;

[0137] The processor 210 is configured to implement the audio data processing method disclosed in any of the above embodiments by running the program stored in the memory 200.

[0138] Specifically, the aforementioned electronic device may also include: a bus, a communication interface 220, an input device 230, and an output device 240.

[0139] The processor 210, memory 200, communication interface 220, input device 230, and output device 240 are interconnected via a bus. Among them:

[0140] A bus can include a pathway for transmitting information between various components of a computer system.

[0141] The processor 210 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0142] Processor 210 may include a main processor, as well as a baseband chip, modem, etc.

[0143] The memory 200 stores a program that executes the technical solution of this invention, and may also store an operating system and other key business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 200 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0144] Input device 230 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0145] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0146] The communication interface 220 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0147] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement the various steps of any of the audio data processing methods provided in the above embodiments of this application.

[0148] For example, the aforementioned electronic devices may specifically be handheld electronic devices, wearable electronic devices, smart terminals, computers, etc., that have audio acquisition and processing functions.

[0149] Exemplary computer program products and storage media

[0150] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps of the audio data processing methods according to various embodiments of this application as described in the "Exemplary Methods" section of this specification.

[0151] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0152] Furthermore, embodiments of this application may also be storage media storing a computer program, which is executed by a processor in the audio data processing methods according to various embodiments of this application described in the "Exemplary Methods" section above. Specifically, the following steps can be implemented:

[0153] Step S101: Acquire multi-channel audio data and determine the number of audio channels required by the target application.

[0154] Step S102: When the number of audio channels is less than the preset number of channels, the multi-channel audio data is processed according to the audio usage scenario of the target application to obtain a first target audio that conforms to the audio usage scenario, and the first target audio is sent to the target application.

[0155] Step S103: If the number of audio channels is not less than the preset number of channels, the multi-channel audio data is sent to the target application so that the target application processes the multi-channel audio data to obtain a second target audio that conforms to the audio usage scenario of the target application.

[0156] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0157] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0158] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0159] The modules and sub-modules in the various embodiments of the present application's devices and terminals can be merged, divided, and deleted according to actual needs.

[0160] It should be understood that the disclosed terminals, devices, and methods can be implemented in other ways, given the several embodiments provided in this application. For example, the terminal embodiments described above are merely illustrative. For instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0161] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0162] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0163] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0164] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0165] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0166] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An audio data processing method, characterized in that, include: Acquire multi-channel audio data and determine the number of audio channels required by the target application; If the number of audio channels is less than the preset number of channels, the multi-channel audio data is processed by a recording system or an audio hardware abstraction layer according to the audio usage scenario of the target application to obtain a first target audio that conforms to the audio usage scenario, and the first target audio is sent to the target application. If the number of audio channels is not less than the preset number of channels, the multi-channel audio data is sent to the target application so that the target application processes the multi-channel audio data to obtain a second target audio that conforms to the audio usage scenario of the target application.

2. The method according to claim 1, characterized in that, The audio usage scenario of the target application is determined in the following way: Obtain the audio usage conditions of the target application, which include the number of audio channels required by the target application for audio data, and the recording source of the audio required by the target application. Based on the audio usage conditions of the target application, determine the audio usage scenario of the target application.

3. The method according to claim 1, characterized in that, The step of processing the multi-channel audio data to obtain a first target audio that conforms to the audio usage scenario of the target application includes: Based on the audio usage scenario, determine the audio processing algorithm corresponding to the audio usage scenario; Using an audio processing algorithm corresponding to the audio usage scenario, the multi-channel audio data is processed to obtain a first target audio that conforms to the audio usage scenario.

4. The method according to claim 3, characterized in that, The step of determining the audio processing algorithm corresponding to the audio usage scenario includes: When the audio usage scenario is audio recording, the audio processing algorithm is determined to be an audio noise reduction algorithm and an audio enhancement algorithm; as well as, When the audio usage scenario is call recording, the audio processing algorithm is determined to be an echo cancellation algorithm.

5. The method according to claim 1, characterized in that, The target application processes the multi-channel audio data in the following manner to obtain a second target audio that matches the audio usage scenario of the target application: The audio processing algorithm corresponding to the audio usage scenario of the target application is invoked to process the multi-channel audio data and obtain a second target audio that conforms to the audio usage scenario of the target application. or, From the audio processing algorithms stored in the target application, an algorithm corresponding to the audio usage scenario of the target application is selected, and the multi-channel audio data is processed to obtain a second target audio that conforms to the audio usage scenario of the target application.

6. The method according to claim 5, characterized in that, The second target audio can be used in at least one of the following scenarios: voice wake-up recognition and voice evaluation.

7. An audio data processing device, characterized in that, include: The first unit is used to acquire multi-channel audio data and determine the number of audio channels required by the target application. The second unit is used to process the multi-channel audio data through a recording system or an audio hardware abstraction layer to obtain a first target audio that conforms to the audio usage scenario, based on the audio usage scenario of the target application, when the number of audio channels is less than the preset number of channels, and then send the first target audio to the target application. The third unit is used to send the multi-channel audio data to the target application when the number of audio channels is not less than the preset number of channels, so that the target application can process the multi-channel audio data to obtain a second target audio that conforms to the audio usage scenario of the target application.

8. An audio data processing system, characterized in that, include: Audio acquisition unit, used to acquire multi-channel audio data; An audio processing unit is used to acquire multi-channel audio data and determine the number of audio channels required by the target application. If the number of audio channels is less than the preset number of channels, the multi-channel audio data is processed by a recording system or an audio hardware abstraction layer according to the audio usage scenario of the target application to obtain a first target audio that conforms to the audio usage scenario, and the first target audio is sent to the target application; if the number of audio channels is not less than the preset number of channels, the multi-channel audio data is sent to the target application so that the target application processes the multi-channel audio data to obtain a second target audio that conforms to the audio usage scenario of the target application.

9. An electronic device, characterized in that, include: Processor and memory; The memory is used to store computer programs; the processor is used to implement the audio data processing method as described in any one of claims 1 to 6 by running the computer programs stored in the memory.

10. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed, implements the audio data processing method according to any one of claims 1-6.

Citation Information

Patent Citations

  • Data forwarding processing method and mobile terminal

    CN106973399A

  • Audio noise processing method and system, electronic device and medium

    CN115223582A