Audio data processing method and device, equipment and storage medium

Abstract

The invention provides an audio data processing method and device, equipment and a storage medium, and relates to the technical field of audio processing. The method comprises the following steps: determining virtual audio equipment corresponding to each audio application program in an application layer; transmitting the audio data to be played to a virtual audio device corresponding to each audio application program in the plurality of virtual audio devices; based on the audio hardware abstraction layer, performing data merging on the to-be-played audio data of the plurality of virtual audio devices, and transmitting the merged to-be-played audio data of the plurality of virtual audio devices to a kernel layer; and transmitting the received audio data to be played of each virtual audio device to the audio coding and decoding chip. According to the method provided by the invention, the requirement of playing multiple paths of different audios at the same time can be realized only by using one audio coding and decoding chip, and the cost is greatly reduced on the hardware level.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and more specifically, to an audio data processing method, apparatus, device, and storage medium. Background Art

[0002] In existing Android audio systems, especially in application scenarios where different audio needs to be played on different speakers, multiple I2S codec chips are usually relied upon to achieve this. For example, in a mobile phone that supports multi-channel surround sound, in order to play audio of different channels independently on their corresponding speakers, multiple I2S codec chips are required to process the audio signals of different channels separately.

[0003] In mass-produced devices, each additional I2S codec chip significantly increases the material cost of the device. The use of multiple I2S codec chips means that more circuit connections and wiring are required, which not only increases the area of ​​the printed circuit board, but also makes the circuit design more complicated, increasing the design cycle and design difficulty. Summary of the Invention

[0004] In response to the above-mentioned deficiencies in the prior art, the present application provides an audio data processing method, apparatus, device and storage medium to solve the problems existing in the prior art.

[0005] The technical solutions adopted in the embodiments of this application are as follows:

[0006] In a first aspect, an embodiment of the present application provides an audio data processing method, which is applied to a processor of a terminal device, wherein the processor is pre-integrated with: an application layer, an audio hardware abstraction layer, and a kernel layer of a preset operating system; the processor is communicatively connected to an audio codec chip; each of multiple interfaces of the audio codec chip is communicatively connected to at least one audio playback device; the method includes:

[0007] Determining a virtual audio device corresponding to each audio application in the application layer;

[0008] Transmitting the audio data to be played to a virtual audio device corresponding to each audio application in the plurality of virtual audio devices;

[0009] Based on the audio hardware abstraction layer, merging the audio data to be played of multiple virtual audio devices, and transmitting the merged audio data to be played of the multiple virtual audio devices to the kernel layer;

[0010] Based on the kernel layer, the received audio data to be played of each virtual audio device is transmitted to the audio codec chip, so that the audio codec chip controls each audio playback device connected to the interface of the communication channel corresponding to each virtual audio device to play the corresponding audio data to be played.

[0011] In one embodiment, determining the virtual audio device corresponding to each audio application in the application layer includes:

[0012] Determining, according to the application program interface of each audio application program and based on a correspondence between a preset application program interface and a virtual audio device, a virtual audio device corresponding to each audio application program;

[0013] The step of transmitting the to-be-played audio data to the virtual audio device corresponding to each audio application in the plurality of virtual audio devices includes:

[0014] According to the application program interface of each audio application program, the audio data to be played is transmitted to the virtual audio device corresponding to each audio application program in the audio hardware abstraction layer.

[0015] In one embodiment, merging the to-be-played audio data of multiple virtual audio devices based on the audio hardware abstraction layer includes:

[0016] Based on the audio hardware abstraction layer, the to-be-played audio data of the multiple virtual audio devices are respectively merged into the buffers of the corresponding communication channels.

[0017] In one embodiment, the step of transmitting the merged audio data to be played of the plurality of virtual audio devices to the kernel layer includes:

[0018] The merged audio data to be played by the multiple virtual audio devices are transmitted to the kernel layer through a preset system audio interface.

[0019] In one embodiment, merging the to-be-played audio data of the multiple virtual audio devices into buffers of corresponding communication channels based on the audio hardware abstraction layer includes:

[0020] Based on the audio hardware abstraction layer, and according to a preset correspondence between virtual audio devices and communication channels, the to-be-played audio data of the multiple virtual audio devices are respectively merged into the buffers of the corresponding communication channels.

[0021] In one embodiment, before merging the to-be-played audio data of the multiple virtual audio devices into the buffers of the corresponding communication channels based on the audio hardware abstraction layer and according to the preset correspondence between the virtual audio devices and the communication channels, the method further includes:

[0022] Based on the audio hardware abstraction layer, a correspondence between the preset virtual audio device and the communication channel is created.

[0023] In one embodiment, each preset virtual audio device corresponds to at least one communication channel.

[0024] In a second aspect, an embodiment of the present application provides an audio data processing device, the audio processing device comprising: an application layer of a preset operating system, an audio hardware abstraction layer, and a kernel layer;

[0025] The application layer is used to determine a virtual audio device corresponding to each audio application in the application layer; and transmit the audio data to be played to the virtual audio device corresponding to each audio application in the plurality of virtual audio devices;

[0026] The audio hardware abstraction layer is used to merge the audio data to be played by multiple virtual audio devices, and transmit the merged audio data to be played by the multiple virtual audio devices to the kernel layer;

[0027] The kernel layer is used to transmit the received audio data to be played of each virtual audio device to the audio codec chip, so that the audio codec chip controls each audio playback device connected to the interface of the communication channel corresponding to each virtual audio device to play the corresponding audio data to be played.

[0028] In a third aspect, an embodiment of the present application provides a terminal device, including a processor, an audio codec chip, and an audio playback device;

[0029] The processor is communicatively connected to the audio codec chip, and the interface of the audio codec chip is communicatively connected to the audio playback device;

[0030] The processor is used to execute program instructions to implement the audio data processing method described in the above embodiment.

[0031] In a fourth aspect, an embodiment of the present application provides a readable storage medium having program instructions stored thereon, and the program instructions, when executed by a processor, implement the audio data processing method described in the above embodiment.

[0032] The beneficial effects of this application are: this application provides an audio data processing method that can realize the demand of playing multiple channels of different audio at the same time by using only one audio codec chip, greatly reducing the cost at the hardware level, reducing the circuit design cost, and being able to meet different personalized needs, with more flexible usage scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0034] Figure 1 A schematic diagram of the principle framework of audio data processing provided in an embodiment of the present application;

[0035] Figure 2 This is a flowchart of an audio data processing method according to an embodiment of the present application;

[0036] Figure 3 The second flowchart of the audio data processing method provided in the embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0038] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative work are within the scope of protection of the present application.

[0039] In addition, the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0040] It should be noted that, in the absence of conflict, the features in the embodiments of this application can be combined with each other.

[0041] The embodiment of the present application provides an audio data processing method, which is applied to a processor of a terminal device, such as Figure 1 As shown, taking the preset operating system as the Android operating system as an example, the processor is pre-integrated with the preset operating system's application layer (Android Framework And Apps), audio hardware abstraction layer (Android Audio HAL), and kernel layer (Linux Kernel). The processor is also communicatively connected to an audio codec chip, which has multiple interfaces, each of which is communicatively connected to at least one audio playback device. The following is a specific example of the audio data processing method provided by this application, combined with the accompanying drawings.

[0042] Figure 2 One of the flow charts of the audio data processing method provided in the embodiment of the present application is as follows: Figure 2 As shown, the method includes:

[0043] S101: Determine a virtual audio device corresponding to each audio application in an application layer.

[0044] First of all, it should be noted that before executing the method of this embodiment, in the audio hardware abstraction layer, the interfaces of multiple communication channels of the audio codec chip are respectively configured as multiple virtual audio devices (Android Audio Device). The audio codec chip is a chip that supports the time division multiplexing protocol (TDM). The time division multiplexing protocol supports 32-channel audio playback. Time division multiplexing refers to the technology of transmitting multiple digital data, voice and video signals in the same device at the same time through cross-bit pulses in different channels or time slots.

[0045] There are multiple audio applications (Apps) in the application layer. When an audio application in the application layer requires audio playback, the user can select the virtual audio device corresponding to the current audio application in the audio application. In other words, the audio playback device to be used to play the audio data to be played by the audio application is determined. If there are multiple audio applications requiring audio playback, the above steps can be repeated to determine the virtual audio device corresponding to each audio application.

[0046] S102: Transmit the audio data to be played to a virtual audio device corresponding to each audio application among the multiple virtual audio devices.

[0047] After determining the virtual audio device corresponding to each audio application, the audio data to be played by each audio application is transferred to the corresponding virtual audio device. The virtual audio device is a preconfigured virtual audio device in the audio hardware abstraction layer. Each virtual audio device corresponds to at least one communication channel of the audio codec chip. Each communication channel corresponds to an interface, and each interface is used to connect to an audio playback device. This application does not limit the number of speakers in the audio playback device. For example, an audio playback device can include one, two, or three speakers.

[0048] S103: Based on the audio hardware abstraction layer, merge the audio data to be played of the multiple virtual audio devices, and transmit the merged audio data to be played of the multiple virtual audio devices to the kernel layer.

[0049] When multiple virtual audio devices of the audio hardware abstraction layer receive the audio data to be played, the audio hardware abstraction layer can merge the audio data to be played of the multiple virtual audio devices. After merging, the audio data to be played of the multiple virtual audio devices are transmitted to the kernel layer through the preset system audio interface (Advanced Linux Sound Architecture, ALSA) in accordance with the format of the transmission protocol of the preset communication channel (that is, the format of the time division multiplexing protocol).

[0050] S104: Based on the kernel layer, the received audio data to be played of each virtual audio device is transmitted to the audio codec chip, so that the audio codec chip controls each audio playback device connected to the interface of the communication channel corresponding to each virtual audio device to play the corresponding audio data to be played.

[0051] Finally, the kernel layer is used to transmit the received audio data to be played of each virtual audio device to the audio codec chip. The audio codec chip can then control the audio playback devices corresponding to each virtual audio device to play the audio data to be played. Among them, each virtual audio device is connected to the audio playback device through the interface corresponding to the communication channel on the audio codec chip.

[0052] To sum up, the embodiment of the present application provides an audio data processing method that can realize the demand of playing multiple channels of different audio at the same time using only one audio codec chip, greatly reducing the cost at the hardware level, reducing the circuit design cost, and being able to meet different personalized needs, with more flexible usage scenarios.

[0053] Figure 3 The second flowchart of the audio data processing method provided in the embodiment of the present application is as follows: Figure 3 As shown, in one embodiment, determining the virtual audio device corresponding to each audio application in the application layer in S101 may include:

[0054] S201: Determine a virtual audio device corresponding to each audio application according to an application program interface of each audio application and a correspondence between a preset application program interface and a virtual audio device.

[0055] Taking an audio application under the Android operating system as an example, after a user selects the virtual audio device corresponding to the current audio application within the audio application, the Android audio application, at the software level, uses the application programming interface (API) and the preset mapping between the API and the virtual audio device to determine the corresponding virtual audio device for the Android audio application. The mapping between the preset API and the virtual audio device is pre-set. Repeating this process for each audio application will determine the corresponding virtual audio device for each audio application.

[0056] The step of transmitting the audio data to be played to the virtual audio device corresponding to each audio application in the multiple virtual audio devices in S102 may include: S202, transmitting the audio data to be played to the virtual audio device corresponding to each audio application in the audio hardware abstraction layer according to the application program interface of each audio application.

[0057] In another embodiment, the step of merging the audio data to be played of multiple virtual audio devices based on the audio hardware abstraction layer in S103 may be merging the audio data to be played of multiple virtual audio devices into buffers of corresponding communication channels based on the audio hardware abstraction layer.

[0058] Specifically, the audio hardware abstraction layer pre-stores the correspondence between the created virtual audio devices and the communication channels of the audio codec chip. The audio hardware abstraction layer can be used to merge the audio data to be played of multiple virtual audio devices into the buffer of the corresponding communication channel according to the preset correspondence between the virtual audio devices and the communication channel, and then transmit the audio data to be played of the multiple virtual audio devices in the buffer to the kernel layer according to the transmission protocol of the preset communication channel.

[0059] In summary, this application provides an audio data processing method. Compared with the traditional solution that usually requires multiple I2S codec chips to play different audio on different audio playback devices, this application implements an innovative method in the Android audio hardware abstraction layer by introducing a time division multiplexing protocol, which has the following advantages:

[0060] 1. By using only one audio codec chip that supports the time division multiplexing protocol, this application can meet the demand of playing multiple different audio channels at the same time, greatly reducing the cost at the hardware level and reducing the circuit design cost.

[0061] 2. The present invention introduces flexible configuration options. By configuring the interfaces of different communication channels of the audio codec chip as audio virtual devices, users can select one of the virtual devices for audio playback through the audio application, which can meet different personalized needs and provide more flexible usage scenarios.

[0062] The following continues to explain the apparatus, device, and storage medium for executing the audio data processing method provided in any of the above embodiments of the present application. The specific implementation process and the technical effects produced are the same as those of the corresponding method embodiments described above. For the sake of brief description, for the parts not mentioned in the following embodiments, please refer to the corresponding content in the method embodiments.

[0063] The present application also provides an audio data processing device, comprising: an application layer of a preset operating system, an audio hardware abstraction layer, and a kernel layer.

[0064] The application layer is used to determine the virtual audio device corresponding to each audio application in the application layer; and transmit the audio data to be played to the virtual audio device corresponding to each audio application in the multiple virtual audio devices.

[0065] The audio hardware abstraction layer is used to merge the audio data to be played by multiple virtual audio devices, and transmit the merged audio data to be played by the multiple virtual audio devices to the kernel layer.

[0066] The kernel layer is used to transmit the received audio data to be played of each virtual audio device to the audio codec chip, so that the audio codec chip controls each audio playback device connected to the interface of the communication channel corresponding to each virtual audio device to play the corresponding audio data to be played.

[0067] The above-mentioned device is used to execute the method provided in the above-mentioned embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0068] The present application also provides a terminal device, including a processor, an audio codec chip and an audio playback device.

[0069] The processor is communicatively connected to the audio codec chip, and the interface of the audio codec chip is communicatively connected to the audio playback device; the processor is used to execute program instructions to implement the audio data processing method described in any of the above embodiments.

[0070] The present application also provides a readable storage medium having program instructions stored thereon. When the program instructions are executed by a processor, the audio data processing method described in any of the above embodiments is implemented.

[0071] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0072] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0073] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0074] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor (English: processor) to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (English: Read-Only Memory, abbreviated: ROM), a random access memory (English: Random Access Memory, abbreviated: RAM), a disk or an optical disk, and other media that can store program code.

[0075] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for processing audio data, characterized in that: A processor applied to a terminal device, the processor pre-integrated with: an application layer of a preset operating system, an audio hardware abstraction layer, and a kernel layer; the processor is communicatively connected to an audio codec chip; each of multiple interfaces of the audio codec chip is communicatively connected to at least one audio playback device; the method comprising: Determining a virtual audio device corresponding to each audio application in the application layer; Transmitting the audio data to be played to a virtual audio device corresponding to each audio application in the plurality of virtual audio devices; Based on the audio hardware abstraction layer, merging the audio data to be played of multiple virtual audio devices, and transmitting the merged audio data to be played of the multiple virtual audio devices to the kernel layer; Based on the kernel layer, the received audio data to be played of each virtual audio device is transmitted to the audio codec chip, so that the audio codec chip controls each audio playback device connected to the interface of the communication channel corresponding to each virtual audio device to play the corresponding audio data to be played.

2. The method according to claim 1, characterized in that The determining of the virtual audio device corresponding to each audio application in the application layer includes: Determining, according to the application program interface of each audio application program and based on a correspondence between a preset application program interface and a virtual audio device, a virtual audio device corresponding to each audio application program; The step of transmitting the to-be-played audio data to the virtual audio device corresponding to each audio application in the plurality of virtual audio devices includes: According to the application program interface of each audio application program, the audio data to be played is transmitted to the virtual audio device corresponding to each audio application program in the audio hardware abstraction layer.

3. The method according to claim 1, characterized in that The step of merging the audio data to be played of multiple virtual audio devices based on the audio hardware abstraction layer includes: Based on the audio hardware abstraction layer, the to-be-played audio data of the multiple virtual audio devices are respectively merged into the buffers of the corresponding communication channels.

4. The method according to claim 1, wherein The step of transmitting the merged audio data to be played by the multiple virtual audio devices to the kernel layer includes: The merged audio data to be played by the multiple virtual audio devices are transmitted to the kernel layer through a preset system audio interface.

5. The method according to claim 3, characterized in that The step of merging the to-be-played audio data of the plurality of virtual audio devices into the buffers of the corresponding communication channels based on the audio hardware abstraction layer includes: Based on the audio hardware abstraction layer, and according to a preset correspondence between virtual audio devices and communication channels, the to-be-played audio data of the multiple virtual audio devices are respectively merged into the buffers of the corresponding communication channels.

6. The method according to claim 5, characterized in that Before merging the to-be-played audio data of the multiple virtual audio devices into the buffers of the corresponding communication channels based on the audio hardware abstraction layer and according to the preset correspondence between the virtual audio devices and the communication channels, the method further includes: Based on the audio hardware abstraction layer, a correspondence between the preset virtual audio device and the communication channel is created.

7. The method according to claim 6, characterized in that Each preset virtual audio device corresponds to at least one communication channel.

8. An audio data processing device, characterized in that: The audio data processing device includes: an application layer of a preset operating system, an audio hardware abstraction layer, and a kernel layer; The application layer is used to determine a virtual audio device corresponding to each audio application in the application layer; and transmit the audio data to be played to the virtual audio device corresponding to each audio application in the plurality of virtual audio devices; The audio hardware abstraction layer is used to merge the audio data to be played by multiple virtual audio devices, and transmit the merged audio data to be played by the multiple virtual audio devices to the kernel layer; The kernel layer is used to transmit the received audio data to be played of each virtual audio device to the audio codec chip, so that the audio codec chip controls each audio playback device connected to the interface of the communication channel corresponding to each virtual audio device to play the corresponding audio data to be played.

9. A terminal device, characterized in that: Including processor, audio codec chip and audio playback device; The processor is communicatively connected to the audio codec chip, and the interface of the audio codec chip is communicatively connected to the audio playback device; The processor is used to execute program instructions to implement the audio data processing method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores program instructions, and the program instructions are executed by a processor to implement the audio data processing method according to any one of claims 1 to 7.

Images