A cross-domain audio processing method, device, electronic device and storage medium

Abstract

The application discloses a cross-domain audio processing method and device, electronic equipment and storage medium, the method is applied to a system on chip, the system on chip comprises at least two operating system domains, the method comprises the following steps: obtaining audio data according to a target virtual device node of a first sound card; wherein the first sound card is located in a first operating system domain; storing the audio data to a target storage position corresponding to the target virtual device node and a first device node of a second sound card in a shared memory according to a preset audio routing relationship; wherein the preset audio routing relationship comprises an association relationship among the target virtual device node, the first device node and a second device node of the second sound card, and the second sound card is located in a second operating system domain; obtaining audio data corresponding to the target storage position according to the first device node; and processing the audio data received by the first device node according to the second device node. The audio data processing in the cross-domain scene is realized, and the user experience is improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a cross-domain audio processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, System-on-a-Chip (SoC) systems in vehicles typically employ a multi-core heterogeneous architecture, with each core running a different operating system. However, there is only one hardware controller within the SoC that manages audio processing. The domain possessing the Audio Direct Memory Access Controller (AUDIO DMAC) resource has control over the audio hardware and can control playback from the inter-ICSound (I2S) hardware integrated within different integrated circuits within the SoC. Domains without DMAC resources cannot directly play audio data, severely degrading the user experience. Summary of the Invention

[0003] This invention provides a cross-domain audio processing method, apparatus, electronic device, and storage medium to enable cross-domain processing of audio data between different operating system domains, maximize the satisfaction of cross-domain audio processing needs, and thereby improve user experience.

[0004] According to one aspect of the present invention, a cross-domain audio processing method is provided, applied to a system-on-a-chip, the system-on-a-chip including at least two operating system domains, the method comprising:

[0005] Audio data is obtained based on the target virtual device node of the first sound card; wherein, the first sound card is located in the first operating system domain;

[0006] According to the preset audio routing relationship, the audio data is stored in the target storage location that is common to the target virtual device node and the first device node of the second sound card in the shared memory; wherein, the preset audio routing relationship includes the association relationship between the target virtual device node, the first device node and the second device node of the second sound card, and the second sound card is located in the second operating system domain;

[0007] Obtain the audio data corresponding to the target storage location based on the first device node;

[0008] The second device node processes the audio data received by the first device node.

[0009] According to another aspect of the present invention, a cross-domain audio processing apparatus is provided, applied to a system-on-a-chip, the system-on-a-chip including at least two operating system domains, the apparatus comprising:

[0010] The first data acquisition module is used to acquire audio data based on the target virtual device node of the first sound card; wherein, the first sound card is located in the first operating system domain;

[0011] The data storage module is used to store audio data to a target storage location in shared memory that corresponds to the target virtual device node and the first device node of the second sound card, according to a preset audio routing relationship. The preset audio routing relationship includes the association between the target virtual device node, the first device node of the second sound card, and the second device node, and the second sound card is located in the second operating system domain.

[0012] The second data acquisition module is used to acquire audio data corresponding to the target storage location based on the first device node;

[0013] The data processing module is used to process the audio data received by the first device node according to the second device node.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the cross-domain audio processing method according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the cross-domain audio processing method according to any embodiment of the present invention.

[0019] The technical solution of this invention involves acquiring audio data based on a target virtual device node of a first sound card, wherein the first sound card is located in a first operating system domain; storing the audio data in a target storage location in shared memory, corresponding to both the target virtual device node and the first device node of the second sound card, according to a preset audio routing relationship; wherein the preset audio routing relationship includes the association between the target virtual device node, the first device node of the second sound card, and the second device node, and the second sound card is located in a second operating system domain; acquiring the audio data corresponding to the target storage location based on the first device node; and processing the audio data received by the first device node based on the second device node. This invention, by using a preset audio routing relationship and shared memory, enables cross-domain transmission of audio data from a target virtual device node in the first operating system domain to a second device node in the second operating system domain for processing, achieving cross-domain processing of audio data from different data sources, maximizing the satisfaction of cross-domain audio processing needs, and thus improving user experience.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a cross-domain audio processing method provided in Embodiment 1 of the present invention;

[0023] Figure 2 This is a flowchart of a cross-domain audio processing method provided according to Embodiment 2 of the present invention;

[0024] Figure 3 This is a flowchart of a second application initialization method provided according to Embodiment 2 of the present invention;

[0025] Figure 4 This is a schematic diagram of a cross-domain audio processing framework provided according to Embodiment 3 of the present invention;

[0026] Figure 5 This is a schematic diagram of the audio routing provided according to Embodiment 3 of the present invention;

[0027] Figure 6 This is a flowchart of a cross-domain audio processing method provided in Embodiment 3 of the present invention;

[0028] Figure 7 This is a schematic diagram of the structure of a cross-domain audio processing device according to Embodiment 4 of the present invention;

[0029] Figure 8 This is a schematic diagram of the structure of an electronic device that implements the cross-domain audio processing method of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] Example 1

[0033] Figure 1 This is a flowchart of a cross-domain audio processing method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where audio data is processed across different operating system domains, such as cross-domain audio playback. This method can be executed by a cross-domain audio processing device, which can be implemented in hardware and / or software. This cross-domain audio processing device can be configured in electronic devices, such as, but not limited to, in-vehicle devices. Figure 1 As shown in the figure, this embodiment provides a cross-domain audio processing method applied to a system-on-a-chip (SOC), wherein the SOC includes at least two operating system domains, and the method specifically includes the following steps:

[0034] S110. Obtain audio data based on the target virtual device node of the first sound card; wherein, the first sound card is located in the first operating system domain.

[0035] The first operating system domain can be understood as the operating system that needs to process audio data across domains. The first operating system domain can include Android domains, Linux domains, and QNX (Quick UNIX) domains, etc. For example, in a cross-domain audio playback scenario, the first operating system domain could be an Android domain that does not have DMAC resources but has audio playback requirements. The first sound card can refer to a virtual sound card device (dummy card) located in the first operating system domain, which can be pre-registered and created in the kernel of the first operating system domain.

[0036] Audio data can refer to audio data to be processed across domains. The audio data can come from different data sources, such as but not limited to: system notification audio, multimedia audio, navigation audio, recording audio, etc. The embodiments of the present invention do not impose such limitations.

[0037] A virtual device node can refer to a series of virtual pulse code modulation (PCM) device nodes created under the first sound card, used for cross-domain processing of audio data in the first operating system domain. The number of virtual device nodes can be configured according to actual needs, and this embodiment of the invention does not limit this.

[0038] The target virtual device node can be understood as the virtual device node currently processing audio data for cross-domain transmission. In essence, multiple target virtual device nodes can be controlled simultaneously to process audio data from different sources in parallel, based on business needs, to achieve synchronous processing of multiple audio data streams in cross-domain scenarios, thereby maximizing the fulfillment of cross-domain audio processing requirements.

[0039] In this embodiment of the invention, the first operating system domain in the SOC can respond to an audio processing application or audio processing command, open the target virtual device node under the first sound card in the first operating system domain, and then obtain the corresponding audio data to be processed across domains from the corresponding data storage location, such as the system local folder or a remote server, according to the data source of the audio data. Then, the audio data is transmitted to the specified target virtual device node by calling a preset interface function or an application programming interface (API).

[0040] Furthermore, corresponding virtual device nodes can be pre-configured for audio data from different data sources. For example, system notification audio uses virtual device node dummy_pcm0 by default, navigation audio uses virtual device node dummy_pcm1 by default, and so on.

[0041] S120. According to the preset audio routing relationship, the audio data is stored in the target storage location that is common to the target virtual device node and the first device node of the second sound card in the shared memory; wherein, the preset audio routing relationship includes the association relationship between the target virtual device node, the first device node and the second device node of the second sound card, and the second sound card is located in the second operating system domain.

[0042] The second operating system domain can be understood as an operating system distinct from the first operating system domain, which needs to process the audio data transmitted from the first operating system domain. The second operating system domain can include Android domains, Linux domains, and QNX (Quick UNIX) domains, etc. Similarly, taking cross-domain audio playback as an example, the second operating system domain can be a Linux domain with DMAC resources. The second sound card can refer to a sound card device located in the second operating system domain, which can be pre-registered and created in the kernel of the second operating system domain.

[0043] The first device node and the second device node are two different types of PCM device nodes created under the second sound card. The first device node is used to read audio data written by the target virtual device node from the shared memory and send the read audio data to the corresponding second device node. The second device node is used to receive the audio data transmitted by the first device node and process it, such as sending the audio data to the corresponding audio controller for audio playback.

[0044] The preset audio routing relationship can be understood as a pre-configured routing relationship used to characterize the transmission path of audio data. The preset audio routing relationship can include the association between the target virtual device node, the first device node of the second sound card, and the second device node.

[0045] Shared memory (SHM) can be understood as a pre-allocated physical address space used for audio data transfer between a first operating system domain and a second operating system domain. Both the first and second operating system domains can access this shared memory. The target storage location refers to the shared memory space shared by the target virtual device node and its corresponding first device node. Essentially, the shared memory can be pre-divided into several storage locations, and associations can be established between the virtual device node under the first sound card, the first device node under the second sound card, and their corresponding storage locations in the shared memory. In this way, the virtual device node and its associated first device node can perform read and write operations on audio data at the specified storage locations in the shared memory.

[0046] In this embodiment of the invention, the SOC can first determine the first device node associated with the target virtual device node according to the pre-configured preset audio routing relationship, then find the target storage area identifier that is common to both the target virtual device node and the first device node in the configuration file or configuration table that stores the shared memory storage location association relationship according to the identifier information of the first device node, then determine the corresponding target storage location in the shared memory based on the above target storage area identifier, and finally control the target virtual device node to write the acquired audio data to be processed across domains to the target storage location.

[0047] S130: Obtain the audio data corresponding to the target storage location based on the first device node.

[0048] In this embodiment of the invention, since there is an association and pairing relationship between the target virtual device node, the first device node, and the target storage location in shared memory, after the target virtual device node writes audio data to the target storage location, the first device node under the second sound card in the second operating system domain can read the corresponding audio data from the specified target storage location. In a specific embodiment, an audio processing thread can be created in the second operating system domain to detect whether the write pointer of the target storage location in shared memory has changed. When a change in the write pointer is detected, the first device node can be controlled to read the audio data in the target storage location, update the read pointer corresponding to the target storage location, and notify the first operating system to continue sending the next frame of audio data.

[0049] S140. Process the audio data received by the first device node according to the second device node.

[0050] In this embodiment of the invention, after the audio data is written to the first device node, the SOC can control the first device node to transmit the audio data to the second device node for subsequent data processing operations. The subsequent data processing operations may include, but are not limited to, the second device node transmitting the acquired audio data to an audio playback hardware player for playback, and the second device node persistently storing the acquired audio data. This embodiment of the invention does not impose any restrictions on these operations.

[0051] The technical solution of this invention involves acquiring audio data based on a target virtual device node of a first sound card, wherein the first sound card is located in a first operating system domain; storing the audio data in a target storage location in shared memory, corresponding to both the target virtual device node and the first device node of the second sound card, according to a preset audio routing relationship; wherein the preset audio routing relationship includes the association between the target virtual device node, the first device node of the second sound card, and the second device node, and the second sound card is located in a second operating system domain; acquiring the audio data corresponding to the target storage location based on the first device node; and processing the audio data received by the first device node based on the second device node. This invention, by using a preset audio routing relationship and shared memory, enables cross-domain transmission of audio data from a target virtual device node in the first operating system domain to a second device node in the second operating system domain for processing, achieving cross-domain processing of audio data from different data sources, maximizing the satisfaction of cross-domain audio processing needs, and thus improving user experience.

[0052] Example 2

[0053] Figure 2 This is a flowchart of a cross-domain audio processing method provided in Embodiment 2 of the present invention. It is further optimized and extended based on the above embodiments and can be combined with various optional technical solutions in the above embodiments. For example... Figure 2 As shown in the figure, the cross-domain audio processing method provided in this embodiment includes the following steps:

[0054] S210. When the first application in the first operating system domain starts, open the target virtual device node of the first sound card and write the audio data to be processed to the target virtual device node.

[0055] The first application may refer to an audio processing application configured in the first operating system domain.

[0056] In this embodiment of the invention, when there is a cross-domain audio processing requirement in the first operating system domain, the first application can be launched, and then the standard interface function in the Advanced Linux Sound Architecture (ALSA) LIB library can be called to open the specified target virtual device node under the first sound card, and the audio data to be processed across domains can be written to the target virtual device node. It should be understood that multiple target virtual device nodes can be opened simultaneously according to actual business needs to process audio data from various different data sources.

[0057] S220. Based on the session identifier of the target virtual device node, find the first device node identifier of the first device node associated with the target virtual device node in the preset audio routing relationship.

[0058] Here, the session identifier (session_id) can refer to the identification information corresponding to the target virtual device node. The first device node identifier can refer to the identification information corresponding to the first device node.

[0059] In this embodiment of the invention, the first device node identifier of the first device node associated with the target virtual device node can be found in the pre-configured preset audio routing relationship according to the session identifier of the target virtual device node and by using data query methods such as keyword matching and fuzzy matching.

[0060] S230. Determine the target storage area identifier that is commonly associated with the target virtual device node and the first device node in the preset storage location configuration file according to the session identifier and the first device node identifier.

[0061] The preset storage location configuration file can be understood as a pre-configured file that stores the association and pairing relationships between the session identifier, the first device node identifier, and the target storage area identifier. The target storage area identifier can refer to the identification information corresponding to the target storage location.

[0062] In this embodiment of the invention, the target storage area identifier corresponding to both the target virtual device node and the first device node can be found in the preset storage location configuration file based on the session identifier and the first device node, so that the corresponding target storage location can be determined according to the target storage area identifier.

[0063] S240: Take the storage area corresponding to the target storage area identifier in the shared memory as the target storage location, and control the target virtual device node to write the audio data to the target storage location.

[0064] In this embodiment of the invention, standard interface functions or custom interface functions in the ALSA LIB library can be called to control the target virtual device node to write the acquired audio data to be processed across domains to the target storage location identified by the target storage area in the shared memory.

[0065] S250, Control the second application in the second operating system domain to open the first and second device nodes associated with the target virtual device node.

[0066] The second application may refer to a background service program configured in the second operating system domain for cross-domain audio processing.

[0067] In this embodiment of the invention, after the second operating system domain completes the kernel audio driver initialization operation, it can run the second application, and then open the first and second device nodes associated with the target virtual device node according to the pre-configured preset audio routing relationship and call the standard interface functions in the ALSA LIB library, in preparation for starting the audio data processing flow in the second operating system domain.

[0068] S260, Call the audio processing thread of the second operating system domain to read the audio data corresponding to the target storage location and write it to the target audio buffer corresponding to the first device node.

[0069] The audio processing thread can refer to a thread created separately for each virtual device node in the second operating system domain to execute cross-domain audio processing control logic. This audio processing thread can be created and generated during the initialization process of the second application.

[0070] An audio buffer can refer to an AudioBuffer, a data buffer provided for audio data in a second operating system domain. Each virtual device node has a corresponding first device node and an audio buffer.

[0071] In this embodiment of the invention, the SOC can call an audio processing thread pre-configured for the target virtual device node in the second operating system domain. In this thread, the kernel function snd_pcm_kernel_read can be called to control the first device node to read the audio data to be processed from the target storage location in shared memory and save the audio data to the target audio buffer corresponding to the first device node.

[0072] S270: Call the audio processing thread to read the audio data in the target audio buffer and write it to the audio controller corresponding to the second device node, and process the audio data in the audio controller.

[0073] The audio controller can be understood as a hardware controller that can transmit audio data to an audio playback device. For example, it may be limited to the AUDIO DMAC controller of the SOC. It is understood that there is only one audio controller in the SOC, which can be assigned to the first operating system domain or the second operating system domain. In this embodiment, the audio controller is assigned to the second operating system domain by default.

[0074] In this embodiment of the invention, after audio data is written to the target audio buffer, the SOC can invoke an audio processing thread to control the second device node corresponding to the first device node to read the audio data in the target audio buffer and send the audio data to an audio controller such as a DMAC. The audio controller then processes the audio data; for example, it can transmit the audio data to an I2S controller, which in turn controls the audio playback device to play the audio data. Furthermore, after a frame of audio data has been processed across domains, the audio processing thread can be invoked to update the read pointer corresponding to the target storage location and notify the target virtual device node of the first operating system to continue sending the next frame of audio data, until all audio data has been processed across domains.

[0075] Furthermore, based on the above embodiments, this second embodiment provides a cross-domain audio processing method that further includes an initialization process for a second application in a second operating system domain. For example... Figure 3 As shown in the figure, the second embodiment of this method for initializing a second application program specifically includes the following steps:

[0076] S310. Call the second application of the second operating system domain to parse the preset audio routing configuration file and obtain the audio routing configuration line; wherein, the audio routing configuration line includes at least the following configuration information: the session identifier of the target virtual device node, the second sound card type, the first device node identifier, and the second device node identifier.

[0077] The preset audio routing configuration file can be understood as a pre-configured configuration file containing audio routing relationships. The preset audio routing configuration file can consist of several lines of data, namely audio routing configuration lines. The specific number of audio routing configuration lines can be configured according to actual business needs, and each audio routing configuration line must include at least the following configuration information: the session identifier (session_id) of the target virtual device node, the second sound card type (card_type), the first device node identifier (src_pcm), and the second device node identifier (dst_pcm).

[0078] In this embodiment of the invention, after the second application in the second operating system domain starts executing the initialization process, the second application will open and parse the preset audio route configuration file in the specified directory to obtain the corresponding audio route configuration line.

[0079] S320. Select the corresponding target virtual device node under the first sound card according to the session identifier, and select the associated and paired first device node and second device node under the corresponding second sound card according to the second sound card type, the first device node identifier and the second device node identifier.

[0080] In this embodiment of the invention, the first and second device nodes associated with the target virtual device node can be determined according to the audio routing configuration information in the audio routing configuration line.

[0081] S330. Connect the selected target virtual device node, first device node and second device node according to each audio routing configuration line to generate the corresponding preset audio routing relationship.

[0082] In this embodiment of the invention, the target virtual device node, the first device node and the second device node can be logically connected according to the audio routing configuration, thereby generating the corresponding preset audio routing relationship.

[0083] S340. Create the corresponding number of audio processing threads in the second operating system domain according to the number of lines in the audio routing configuration.

[0084] In this embodiment of the invention, a corresponding number of audio processing threads can be created in the second operating system domain according to the number of audio routing configuration lines, that is, an audio processing thread for executing cross-domain audio processing control logic can be created for each virtual device node.

[0085] Furthermore, based on the above embodiments, this second embodiment provides a cross-domain audio processing method that, before writing the audio data to be processed to the target virtual device node, further includes the following steps:

[0086] A. Transmitting an audio processing request message corresponding to the audio data to a second application in the second operating system domain via a remote inter-core communication bus in the first operating system domain; wherein, the audio processing request message includes audio playback parameters;

[0087] B. After the second application determines that the audio playback parameters meet the preset hardware playback requirements, it sends a check message to the first operating system domain.

[0088] The Remote Processor Message (RPMSG) can be understood as a message bus. The kernel driver contains a set of communication interfaces, which can be called in multi-core heterogeneous processors to achieve communication between different CPU cores. An audio processing request message can refer to a request message sent from a first operating system domain to a second operating system domain for cross-domain audio data processing. Audio playback parameters may include, but are not limited to, the sampling rate, number of channels, and bit depth of the audio to be processed.

[0089] In this embodiment of the invention, before writing the audio data to be processed to the target virtual device node, i.e., before performing cross-domain processing of the audio data, the kernel of the first operating system domain can send an audio processing request message corresponding to the audio data to be processed across domains to the second application via the RPMSG mechanism. The audio playback parameters may include, but are not limited to, the sampling rate, number of channels, and sampling bit depth of the audio to be processed. After receiving the audio processing request message via the RPMSG mechanism, the second application will check whether the corresponding audio playback parameters meet the preset hardware playback requirements. If the parameters meet the requirements, it will send a correct check message back to the first operating system domain via the RPMSG mechanism to start executing the next step of audio data processing logic. If the parameters do not meet the requirements, the first operating system domain will directly return an error message.

[0090] Furthermore, based on the above embodiments, the cross-domain audio processing method provided in this second embodiment further includes:

[0091] Register a first sound card in a first operating system domain and create a target virtual device node under the first sound card; and register a second sound card in a second operating system domain and create an associated and paired first device node and second device node under the second sound card, wherein the type of the second sound card includes at least: integrated circuit built-in audio bus type and time division multiplexing type.

[0092] In this embodiment of the invention, after the kernel audio driver in the first operating system domain and the second operating system domain is initialized, the kernel standard function is called to register the first sound card in the first operating system domain and the second sound card in the second operating system domain, respectively. Then, according to the actual business needs, a preset number of target virtual device nodes are created under the first sound card and a preset number of associated and paired first and second device nodes are created under the second sound card. The type of the second sound card includes at least: integrated circuit built-in audio bus type (i2s) and time division multiplexing type (TDM).

[0093] It's worth noting that when the second sound card is of the I2S type, the corresponding I2S PCM device node exclusively occupies one I2S hardware resource, suitable for scenarios involving single-channel playback. When the second sound card is of the TDM type, one I2S hardware resource can be expanded into a preset number of second device nodes via software, enabling simultaneous playback of different audio data using only one I2S hardware, suitable for multi-region scenarios. Both methods can work simultaneously in cross-domain scenarios, meeting all cross-domain audio playback needs.

[0094] The technical solution of this invention involves the following steps: When a first application in a first operating system domain starts, it opens the target virtual device node of the first sound card and writes the audio data to be processed to the target virtual device node; it searches for the first device node identifier of the first device node associated with the target virtual device node in a preset audio routing relationship according to the session identifier of the target virtual device node; it determines the target storage area identifier commonly corresponding to the target virtual device node and the first device node in a preset storage location configuration file according to the session identifier and the first device node identifier; it uses the storage area in shared memory corresponding to the target storage area identifier as the target storage location and controls the target virtual device node to write the audio data to the target storage location; it controls a second application in a second operating system domain to open the first device node and the second device node associated with the target virtual device node; it calls the audio processing thread of the second operating system domain to read the audio data corresponding to the target storage location and write it to the target audio buffer corresponding to the first device node; it calls the audio processing thread to read the audio data in the target audio buffer and write it to the audio controller corresponding to the second device node, and processes the audio data in the audio controller. This invention enables cross-domain processing of audio data from different data sources. By flexibly configuring I2S hardware resources, it can maximize the satisfaction of cross-domain audio processing needs, thereby improving user experience and enhancing the scalability and flexibility of audio processing services in cross-domain scenarios.

[0095] Example 3

[0096] Figure 4 This is a schematic diagram of a cross-domain audio processing framework provided in Embodiment 3 of the present invention. Figure 4 As shown, the framework includes a multi-core heterogeneous SOC, whose internal hardware is divided into an Android domain and a Linux domain. These two domains are completely isolated in terms of hardware. Each domain has a user space and a kernel space. The user space is for application processing programs, and the kernel space is for low-level drivers. The two are connected through system calls.

[0097] In the Android domain, the audio process is the application. DUMMY-PCM is a PCM device node created under the dummy card implemented in the underlying driver. This is a virtual PCM device node created under a virtual sound card, used to play audio data for the Android domain. In the Linux domain, the audio thread is an audio processing thread created in user space.

[0098] In TDM-PCM, FE stands for Front End, and BE stands for Back End. FE and BE are implemented based on the Linux kernel's Dynamic Pulse Code Modulation (DPCM) framework. Each FE corresponds to one PCM device, such as... Figure 4 In this context, fe0 corresponds to pcm0, fe1 corresponds to pcm1, and so on. All fe in a TDM are connected to beta, and the beta are connected to the TDM hardware device through a Digital Audio Interface (DAI). The Audio Dmac is the audio DMA controller, a hardware module used to store and transmit audio data.

[0099] In a Linux domain, I2S-PCM refers to PCM device nodes created by the sound card under ALSA. Each PCM device node under this sound card corresponds to a specific I2S hardware path. TDM-PCM refers to PCM device nodes created by the TDM card under ALSA. Each PCM device node under this sound card also corresponds to a specific I2S hardware path. The key difference is that the I2S hardware path for this path cannot be the same as that for the I2S-PCM. Furthermore, TDM-PCM is implemented as multiple PCM device nodes in the kernel driver through software, enabling simultaneous playback of multiple audio sources in cross-domain scenarios. SHARE-PCM is used to retrieve audio data from the domain to be played from shared memory and then saves the retrieved audio data to the audio buffer corresponding to each SHARE-PCM. SHARE-PCM can transmit audio data to TDM-PCM or I2S-PCM via the Audio Buffer. The audio routing pairing is configured by parsing the audio routing configuration file audio_config.xml in the Linux domain. I2S-PCM and TDM-PCM read the required audio data from the Audio Buffer and finally write the audio data to the DMAC.

[0100] like Figure 5 As shown, a dummy PCM is created under the Dummy Card in the Android domain, a share_pcm and an i2s_pcm are created under the Sound Card in the Linux domain, and a share_pcm and a tdm_pcm are created under the Tdm Card in the Linux domain. There is a one-to-one correspondence between the DUMMY-PCM in the Android domain and the SHARE-PCM, I2S-PCM, and TDM-PCM in the Linux domain. Below are two examples illustrating this:

[0101] Example 1: Audio data is played in the Android domain through the dummy pcm0 device node of DUMMY-PCM. At this time, the share_i2s_pcm0 device node under SHARE-PCM in the Linux domain obtains the audio data and transmits the audio data to i2s_pcm0 under I2S-PCM. Finally, i2s_pcm0 writes the audio data to DMAC.

[0102] Example 2: In the Android domain, audio data is played through the dummy pcm6 device node of DUMMY-PCM. At this time, the share_tdm_pcm0 node under SHARE-PCM in the Linux domain obtains the audio data and transmits the audio data to tdm_pcm0 under TDM-PCM. Finally, tdm_pcm0 writes the audio data to DMAC.

[0103] As explained above, in a domain with DMAC resource hardware, two ALSA sound card devices are created. The first ALSA sound card device implements I2S-PCM and SHARE-PCM, and the second ALSA sound card device implements TDM-PCM and SHARE-PCM. In a domain that does not have DMAC resources but needs to transmit audio data across domains, one ALSA sound card device is created, and DUMMY-PCM is implemented under this sound card. TDM-PCM and I2S-PCM are functionally independent. I2S-PCM corresponds to one physical I2S hardware, with only one PCM device node, and can only play one audio data stream at a time. TDM-PCM, through software, expands one I2S hardware into multiple PCM device nodes, each of which can play different audio data simultaneously. Both TDM-PCM and I2S-PCM can work simultaneously in cross-domain scenarios, maximizing the satisfaction of audio playback needs in various cross-domain scenarios.

[0104] Based on the aforementioned cross-domain audio processing framework Figure 6 This is a flowchart of a cross-domain audio processing method provided in Embodiment 3 of the present invention. Based on the above embodiments, this embodiment takes Android as the first operating system domain and Linux as the second operating system domain as an example, providing one implementation of the cross-domain audio processing method, which can realize cross-domain playback of audio data. Figure 6 As shown, the cross-domain audio processing method provided in Embodiment 3 of the present invention specifically includes the following steps:

[0105] S410, Perform initialization operations on the Android domain kernel.

[0106] In this embodiment of the invention, S410 specifically includes the following steps:

[0107] S4101. After the system is powered on, the Android domain kernel audio driver is initialized.

[0108] S4102, Register the first sound card under ALSA.

[0109] Specifically, the dummy card sound card, i.e. the first sound card, can be registered by calling the ALSA standard function devm_snd_soc_register_card in the Android domain kernel.

[0110] S4103. Create a virtual device node under the first sound card.

[0111] Specifically, the ALSA standard function soc_new_pcm can be called in the Android domain kernel to create a preset number of dummy PCM device nodes, i.e., virtual device nodes, under the first sound card dummy card. The number of PCM device nodes is specified according to actual needs. For example, 8 PCM device nodes can be created for data playback with the Linux domain.

[0112] S4104. Initialize shared memory.

[0113] Specifically, a 2MB physical shared memory segment can be initialized. This 2MB physical memory is used to transfer large amounts of audio data between the Android and Linux domains. It should be understood that the use of 2MB of shared memory described above is merely an example; other sizes of shared memory can be used in practical applications, and this embodiment does not impose any specific limitations on this.

[0114] S4105. Initialize the rpmsg driver, create rpmsg callback functions for interaction with the Linux domain application layer, and create rpmsg callback functions for interaction with the Linux domain kernel layer.

[0115] At this point, the Android domain underlying driver initialization is complete.

[0116] S420 performs initialization operations on the Linux domain kernel.

[0117] In this embodiment of the invention, S420 specifically includes the following steps:

[0118] S4201. After the system is powered on, the Linux domain kernel audio driver is initialized.

[0119] S4202, Register the second sound card of type i2s under ALSA.

[0120] Specifically, a sound card, i.e., an I2S type secondary sound card, can be registered by calling the ALSA standard function devm_snd_soc_register_card in the Linux domain kernel.

[0121] S4203, For i2s type second sound card, create associated pairing of first device node and second device node under the second sound card.

[0122] Specifically, the ALSA standard function `soc_new_pcm` can be called in the Linux domain kernel to create a preset number of associated paired first device nodes (shared PCM device nodes) and second device nodes (i2s PCM device nodes) under an I2S type second sound card. The number of i2s PCM nodes is determined by the SOC hardware; for example, if the SOC has 8 I2S hardware channels, then 8 i2s PCM device nodes can be registered. In this embodiment, at least one I2S hardware channel is reserved for the TDMcard. The first device node (shared PCM device node) and the second device node (i2s PCM device node) have a one-to-one correspondence. For example, creating three i2s PCM device nodes `i2s_pcm0`, `i2s_pcm1`, and `i2s_pcm2` under the sound card will simultaneously create three corresponding shared PCM device nodes `share_i2s_pcm0`, `share_i2s_pcm1`, and `share_i2s_pcm2`. For example, audio data in shared memory can be obtained through share_pcm0 and written to i2s_pcm0 via Audio Buffer.

[0123] S4204, Register a second sound card of type TDM under ALSA.

[0124] Similarly, a TDM card sound card, i.e., a second TDM type sound card, can be registered by calling the ALSA standard function devm_snd_soc_register_card in the Linux domain kernel.

[0125] S4205. For a second sound card of type TDM, create an associated pairing of a first device node and a second device node under the second sound card.

[0126] Similarly, the ALSA standard function `soc_new_pcm` can be called in the Linux domain kernel to create a preset number of associated paired first device nodes (shared PCM device nodes) and second device nodes (TDM PCM device nodes) under a TDM-type second sound card. TDM PCM uses one I2S hardware channel to generate multiple PCM device nodes via software. The number of TDM PCM device nodes depends on the SOC hardware. For example, if the I2S is configured in TDM mode and the hardware supports 16 slots, then a maximum of 8 PCM device nodes can be generated as needed. Each generated PCM device node corresponds to two slots in the TDM; for example, pcm0 can correspond to slots 1 and 2, pcm1 can correspond to slots 3 and 4, and so on. The multiple TDM PCM device nodes implemented in software are independent, and each PCM can play one channel of two-channel audio data. Simultaneously, TDM PCM0 can play up to 16 channels of audio data, meeting the playback requirements of multi-channel audio data in cross-domain scenarios. There is a one-to-one correspondence between the two types of PCMs: the first device node (shared PCM device node) and the second device node (TDM PCM device node). For example, creating three TDM PCM device nodes (tdm_pcm0, tdm_pcm1, and tdm_pcm2) under a TDM card will simultaneously create three corresponding shared PCM device nodes (share_tdm_pcm0, share_tdm_pcm1, and share_tdm_pcm2). For instance, audio data can be retrieved from shared memory through shared_tdm_pcm0 and written to tdm_pcm0 via an Audio Buffer.

[0127] S4206. Initialize shared memory.

[0128] Specifically, a 2MB physical shared memory segment can be initialized. This 2MB physical memory segment is the same memory address as the one in step S4104, and both the Linux domain and the Android domain will use this memory address.

[0129] S4207. Initialize the rpmsg driver and create an rpmsg callback function that interacts with the Android domain kernel layer.

[0130] S4208. Create a PCM data processing thread in the Linux domain kernel and wait for it to run.

[0131] At this point, the Linux domain underlying driver initialization is complete.

[0132] S430 performs initialization operations on the second application in the Linux domain.

[0133] In this embodiment of the invention, S430 specifically includes the following steps:

[0134] S4301, the second application of the Linux domain begins to run; this is a background service program.

[0135] S4302: Call the second application to parse the preset audio route configuration file and obtain the audio route configuration line.

[0136] Specifically, a second application can be invoked to open and parse the audio_config.xml file in the specified directory, which is the default audio routing configuration file. The audio_config.xml file consists of several lines of audio routing configuration lines, and the specific number of audio routing configuration lines can be configured according to actual business needs. Each audio routing configuration line follows a fixed format of session_id / card_type / src_pcm / dst_pcm, representing a valid audio route. Among them, session_id represents the session identifier, corresponding to the value of dummy pcm; card_type corresponds to either sound card or TDM card; src_pcm and dst_pcm are integer variables defined in the XML, src_pcm (user side) corresponds to the kernel's share_pcm, and dst_pcm corresponds to the kernel's i2s_pcm or TDM_pcm.

[0137] S4303. Create the corresponding number of audio processing threads according to the audio routing configuration line.

[0138] Specifically, based on the src pcm, dst pcm, session id, and card_type parsed in step S4302, an audio processing thread can be created for each session id. This thread is used to handle the control logic of the corresponding src pcm (shared pcm) and dst pcm (i2spcm or tdm pcm).

[0139] S4304. Create an rpmsg device node in the second application that interacts with the Android domain kernel.

[0140] At this point, the initialization of the second application in the Linux domain is complete.

[0141] The first application in the S440 Android domain starts running, ready to play audio data.

[0142] S450: Open the target virtual device node of the first sound card, write the audio data to be processed to the target virtual device node, and control the target virtual device node to write the audio data to the target storage location in shared memory.

[0143] Specifically, the Hardware Abstraction Layer (HAL) in the Android domain calls the alsalib standard function `pcm_open` to open the target virtual device node (dummy pcm device node) specified under the first sound card's dummy card. In the Android HAL layer, multiple dummy pcm device nodes can be opened simultaneously to play different audio data sources according to business needs. When the `pcm_open` function executes, it sends audio parameters such as the sampling rate, number of channels, and bit depth of the audio to be played to a second application in the Linux domain via the Android domain kernel's rpmsg mechanism. After receiving the audio parameters via the rpmsg mechanism, the second application in the Linux domain checks whether the audio parameters meet the hardware playback requirements. If the parameters meet the requirements, it sends a correct message to the Android domain kernel via the rpmsg mechanism. After receiving the correct message, the Android domain kernel begins processing the next step of the logic. The Android HAL layer continues to call the standard function `pcm_write` to write the audio data to be played to the target virtual device node. If the parameters do not meet the requirements, the Android domain kernel will directly return an error message.

[0144] S460: Control the second application to open the first and second device nodes associated with the target virtual device node.

[0145] Specifically, after the second application in the Linux domain checks that the audio parameters are correct, it calls the alsa lib standard function pcm_open to open the src_pcm and dst_pcm device nodes respectively, which are the first and second device nodes associated with the target virtual device node, and then calls the pcm_start function to prepare for processing the audio data.

[0146] S470, Call the audio processing thread to read the audio data corresponding to the target storage location and write it to the target audio buffer corresponding to the first device node.

[0147] Specifically, after calling the `pcm_start` function in step S460, both the first device node `src_pcm` and the second device node `dst_pcm` begin running. This triggers the execution of the Linux domain's underlying PCM data processing thread (corresponding to the audio processing thread in the Linux domain user space) created in step S4208. Within this thread, the kernel function `snd_pcm_kernel_read` is called on the first device node `share_pcm` (which is `src_pcm` in the user space) to read audio data from `shm` and save it to the `AudioBuffer`. The audio data in `shm` here was written to shared memory by the Android domain through the target virtual device node (dummy PCM device node) in step S450, and is retrieved here through the first device node `share_pcm`. Each target virtual device node has a corresponding first device node `share_pcm` and a target audio buffer.

[0148] S480: Call the audio processing thread to read the audio data in the target audio buffer and write it to the audio controller corresponding to the second device node, and process the audio data in the audio controller.

[0149] Specifically, in the Linux domain kernel PCM data processing thread, the kernel function snd_pcm_kernel_write is called for the second device node (i2spcm or tdm pcm) to read audio data from the Audio Buffer and save it to the audio dma buffer, and then write the audio data in the audio dma buffer to the hardware DMAC (i.e., audio controller).

[0150] S490: Call the audio processing thread to update the read pointer in the shared memory and notify the first operating system to continue sending the next frame of audio data until all the audio data in the Android domain has been sent to the Linux domain for processing.

[0151] Specifically, after the operation in step S480 is completed, the Linux domain PCM data processing thread updates the read pointer in shm and sends a message to the Android domain kernel through the Linux domain kernel rpmsg mechanism. After receiving the message, the Android domain kernel's rpmsg callback function parses the message, updates the write pointer in shm, and starts sending the next frame of audio data until all the audio data of the Android domain has been processed across domains.

[0152] The technical solution of this invention involves performing initialization operations on the Android domain kernel; performing initialization operations on the Linux domain kernel; performing initialization operations on the second application in the Linux domain; the first application in the Android domain starts running and prepares to play audio data; opening the target virtual device node of the first sound card and writing the audio data to be processed to the target virtual device node, and controlling the target virtual device node to write the audio data to the target storage location in shared memory; controlling the second application to open the first device node and the second device node associated with the target virtual device node; calling the audio processing thread to read the audio data corresponding to the target storage location and write it to the target audio buffer corresponding to the first device node; calling the audio processing thread to read the audio data in the target audio buffer and write it to the audio controller corresponding to the second device node, and processing the audio data in the audio controller; calling the audio processing thread to update the read pointer in shared memory and notifying the first operating system to continue sending the next frame of audio data, until all the audio data in the Android domain has been sent to the Linux domain for processing. This invention allows for flexible configuration of I2S hardware resources for playback in cross-domain scenarios. Simultaneously, it enables playback across multiple PCM device nodes via a single I2S hardware connection, maximizing the utilization of available system hardware resources. This meets the needs of multi-functional audio services, enhances user experience, and saves on hardware design costs. Furthermore, it improves the scalability and flexibility of the program in cross-domain scenarios. All audio data processing is implemented in kernel mode, with user mode handling only control command-related logic, significantly reducing audio latency.

[0153] Example 4

[0154] Figure 7 This is a schematic diagram of a cross-domain audio processing device provided in Embodiment 4 of the present invention. Figure 7 As shown, a cross-domain audio processing device is applied to a system-on-a-chip (SoC), which includes at least two operating system domains. The device includes:

[0155] The first data acquisition module 51 is used to acquire audio data based on the target virtual device node of the first sound card; wherein the first sound card is located in the first operating system domain;

[0156] The data storage module 52 is used to store audio data to a target storage location in shared memory that corresponds to the target virtual device node and the first device node of the second sound card, according to a preset audio routing relationship; wherein, the preset audio routing relationship includes the association between the target virtual device node, the first device node of the second sound card, and the second device node, and the second sound card is located in the second operating system domain;

[0157] The second data acquisition module 53 is used to acquire audio data corresponding to the target storage location based on the first device node;

[0158] The data processing module 54 is used to process the audio data received by the first device node according to the second device node.

[0159] The technical solution of this invention involves a first data acquisition module acquiring audio data based on a target virtual device node of a first sound card, wherein the first sound card is located in a first operating system domain; a data storage module storing the audio data in shared memory to a target storage location corresponding to both the target virtual device node and the first device node of the second sound card, according to a preset audio routing relationship; wherein the preset audio routing relationship includes the association between the target virtual device node, the first device node of the second sound card, and the second device node, and the second sound card is located in a second operating system domain; a second data acquisition module acquiring the audio data corresponding to the target storage location based on the first device node; and a data processing module processing the audio data received by the first device node based on the second device node. This invention, by using a preset audio routing relationship and shared memory, enables cross-domain transmission of audio data from a target virtual device node in the first operating system domain to a second device node in the second operating system domain for processing, achieving cross-domain processing of audio data from different data sources, maximizing the satisfaction of cross-domain audio processing needs, and thus improving user experience.

[0160] Furthermore, based on the above embodiments of the invention, the first data acquisition module 51 includes:

[0161] The first data writing unit is used to open the target virtual device node of the first sound card and write the audio data to be processed to the target virtual device node when the first application in the first operating system domain starts.

[0162] Furthermore, based on the above embodiments of the invention, the data storage module 52 includes:

[0163] The node identifier lookup unit is used to look up the first device node identifier of the first device node associated with the target virtual device node in the preset audio routing relationship according to the session identifier of the target virtual device node;

[0164] The storage identifier determination unit is used to determine the target storage area identifier that is commonly corresponding to the target virtual device node and the first device node in the preset storage location configuration file according to the session identifier and the first device node identifier;

[0165] The second data writing unit is used to take the storage area identified by the target storage area in the shared memory as the target storage location, and control the target virtual device node to write the audio data to the target storage location.

[0166] Furthermore, based on the above embodiments, the second data acquisition module 53 includes:

[0167] A node opening unit is used to control a second application in the second operating system domain to open a first device node and a second device node associated with the target virtual device node.

[0168] The third data writing unit is used to call the audio processing thread of the second operating system domain to read the audio data corresponding to the target storage location and write it to the target audio buffer corresponding to the first device node.

[0169] Furthermore, based on the above embodiments of the invention, the data processing module 54 includes:

[0170] The data processing unit is used to call the audio processing thread to read the audio data in the target audio buffer and write it to the audio controller corresponding to the second device node, and then process the audio data in the audio controller.

[0171] Furthermore, based on the above embodiments of the invention, the cross-domain audio processing apparatus further includes:

[0172] The file parsing module is used to call the second application of the second operating system domain to parse the preset audio routing configuration file and obtain the audio routing configuration line; wherein, the audio routing configuration line includes at least the following configuration information: the session identifier of the target virtual device node, the second sound card type, the first device node identifier, and the second device node identifier;

[0173] The device node selection module is used to select the corresponding target virtual device node under the first sound card according to the session identifier, and to select the associated and paired first device node and second device node under the corresponding second sound card according to the second sound card type, the first device node identifier and the second device node identifier.

[0174] The audio route generation module is used to connect the selected target virtual device node, the first device node, and the second device node according to each audio route configuration line to generate the corresponding preset audio route relationship.

[0175] The thread creation module is used to create a corresponding number of audio processing threads in the second operating system domain according to the number of lines in the audio routing configuration.

[0176] Furthermore, based on the above embodiments of the invention, the cross-domain audio processing apparatus further includes:

[0177] An audio processing request module is used to transmit an audio processing request message corresponding to the audio data to a second application in a second operating system domain via a remote inter-core communication bus before writing the audio data to be processed to the target virtual device node; wherein, the audio processing request message includes audio playback parameters;

[0178] The message feedback module is used to send a check message to the first operating system domain after the second application determines that the audio playback parameters meet the preset hardware playback requirements.

[0179] Furthermore, based on the above embodiments of the invention, the cross-domain audio processing apparatus further includes:

[0180] The sound card and device node creation module is used to register a first sound card in a first operating system domain and create a target virtual device node under the first sound card; and to register a second sound card in a second operating system domain and create an associated and paired first device node and second device node under the second sound card, wherein the type of the second sound card includes at least: integrated circuit built-in audio bus type and time division multiplexing type.

[0181] Furthermore, based on the above embodiments of the invention, when the second sound card is of the time-division multiplexing type, the hardware of the built-in audio bus of one integrated circuit is expanded into a preset number of second device nodes by means of software.

[0182] The cross-domain audio processing apparatus provided in the embodiments of the present invention can execute the cross-domain audio processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0183] Example 5

[0184] Figure 8 A schematic diagram of an electronic device 60 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0185] like Figure 8As shown, the electronic device 60 includes at least one processor 61 and a memory, such as a read-only memory (ROM) 62 and a random access memory (RAM) 63, communicatively connected to the at least one processor 61. The memory stores computer programs executable by the at least one processor. The processor 61 can perform various appropriate actions and processes based on the computer program stored in the ROM 62 or loaded into the RAM 63 from storage unit 68. The RAM 63 may also store various programs and data required for the operation of the electronic device 60. The processor 61, ROM 62, and RAM 63 are interconnected via a bus 64. An input / output (I / O) interface 65 is also connected to the bus 64.

[0186] Multiple components in electronic device 60 are connected to I / O interface 65, including: input unit 66, such as keyboard, mouse, etc.; output unit 67, such as various types of monitors, speakers, etc.; storage unit 68, such as disk, optical disk, etc.; and communication unit 69, such as network card, modem, wireless transceiver, etc. Communication unit 69 allows electronic device 60 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0187] Processor 61 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 61 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 61 performs the various methods and processes described above, such as cross-domain audio processing methods.

[0188] In some embodiments, the cross-domain audio processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 68. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 60 via ROM 62 and / or communication unit 69. When the computer program is loaded into RAM 63 and executed by processor 61, one or more steps of the cross-domain audio processing method described above may be performed. Alternatively, in other embodiments, processor 61 may be configured to perform the cross-domain audio processing method by any other suitable means (e.g., by means of firmware).

[0189] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0190] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0191] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0192] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0193] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0194] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0195] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0196] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A cross-domain audio processing method, characterized in that, Applied to a system-on-a-chip, wherein the system-on-a-chip includes at least two operating system domains, the method includes: Audio data is obtained based on the target virtual device node of the first sound card; wherein, the first sound card is located in the first operating system domain; The audio data is stored in a target storage location in shared memory, corresponding to both the target virtual device node and the first device node of the second sound card, according to a preset audio routing relationship. The preset audio routing relationship includes the association between the target virtual device node, the first device node of the second sound card, and the second device node of the second sound card, with the second sound card located in a second operating system domain. The target virtual device node refers to a series of virtual pulse-code modulation device nodes created under the first sound card. The first device node and the second device node are two different types of device nodes created under the second sound card. The first device node reads the audio data written by the target virtual device node from the shared memory and sends the read audio data to the corresponding second device node. The second device node receives the audio data transmitted by the first device node. The target storage location refers to the storage space in shared memory where the target virtual device node and the corresponding first device node are jointly associated. The audio data corresponding to the target storage location is obtained based on the first device node; The second device node processes the audio data received by the first device node. The method further includes: The second application in the second operating system domain is invoked to parse the preset audio routing configuration file to obtain the audio routing configuration line; wherein, the audio routing configuration line includes at least the following configuration information: the session identifier of the target virtual device node, the second sound card type, the first device node identifier, and the second device node identifier; Select the corresponding target virtual device node under the first sound card according to the session identifier, and select the associated and paired first device node and second device node under the corresponding second sound card according to the second sound card type, the first device node identifier and the second device node identifier; According to each of the audio routing configuration lines, the selected target virtual device node, the first device node, and the second device node will be connected to generate the corresponding preset audio routing relationship; Create a corresponding number of audio processing threads in the second operating system domain according to the number of rows in the audio routing configuration.

2. The method according to claim 1, characterized in that, The step of obtaining audio data based on the target virtual device node of the first sound card includes: When the first application in the first operating system domain starts, the target virtual device node of the first sound card is opened, and the audio data to be processed is written to the target virtual device node.

3. The method according to claim 1, characterized in that, The step of storing the audio data to the target storage location located in shared memory, corresponding to the target virtual device node and the first device node of the second sound card, according to a preset audio routing relationship includes: According to the session identifier of the target virtual device node, the first device node identifier associated with the first device node is searched in the preset audio routing relationship; Based on the session identifier and the first device node identifier, determine the target storage area identifier that is commonly corresponding to the target virtual device node and the first device node in the preset storage location configuration file; The storage region in the shared memory corresponding to the target storage region identifier is taken as the target storage location, and the target virtual device node is controlled to write the audio data to the target storage location.

4. The method according to claim 1, characterized in that, The step of obtaining the audio data corresponding to the target storage location based on the first device node includes: The second application in the second operating system domain controls the opening of the first device node and the second device node associated with the target virtual device node; The audio processing thread of the second operating system domain is invoked to read the audio data corresponding to the target storage location and write it to the target audio buffer corresponding to the first device node.

5. The method according to claim 4, characterized in that, The step of processing the audio data received by the first device node according to the second device node includes: The audio processing thread is invoked to read the audio data in the target audio buffer and write it to the audio controller corresponding to the second device node, and the audio data is processed in the audio controller.

6. The method according to claim 2, characterized in that, Before writing the audio data to be processed to the target virtual device node, the method further includes: In the first operating system domain, an audio processing request message corresponding to the audio data is transmitted to a second application in the second operating system domain via a remote inter-core communication bus; wherein, the audio processing request message includes audio playback parameters; After the second application determines that the audio playback parameters meet the preset hardware playback requirements, it sends a correct check message back to the first operating system domain.

7. The method according to claim 1, characterized in that, Also includes: Register the first sound card in the first operating system domain, and create the target virtual device node under the first sound card; In addition, the second sound card is registered in the second operating system domain, and the first device node and the second device node are associated and paired under the second sound card, wherein the type of the second sound card includes at least: integrated circuit built-in audio bus type and time division multiplexing type.

8. The method according to claim 1, characterized in that, When the second sound card is a time-division multiplexed type, the hardware of the built-in audio bus of one integrated circuit is expanded into a preset number of second device nodes through software.

9. A cross-domain audio processing device, characterized in that, Applied to a system-on-a-chip, the system-on-a-chip including at least two operating system domains, the device includes: The first data acquisition module is used to acquire audio data based on the target virtual device node of the first sound card; wherein, the first sound card is located in the first operating system domain; A data storage module is used to store the audio data to a target storage location in shared memory that corresponds to both the target virtual device node and the first device node of the second sound card, according to a preset audio routing relationship. The preset audio routing relationship includes the association between the target virtual device node, the first device node of the second sound card, and the second device node of the second sound card, with the second sound card located in a second operating system domain. The target virtual device node refers to a series of virtual pulse-code modulation device nodes created under the first sound card. The first device node and the second device node are two different types of device nodes created under the second sound card. The first device node is used to read audio data written by the target virtual device node from the shared memory and send the read audio data to the corresponding second device node. The second device node is used to receive audio data transmitted by the first device node. The target storage location refers to the storage space in shared memory that is jointly associated with the target virtual device node and the corresponding first device node. The second data acquisition module is used to acquire the audio data corresponding to the target storage location based on the first device node; The data processing module is used to process the audio data received by the first device node according to the second device node; The cross-domain audio processing device further includes: The file parsing module is used to call the second application of the second operating system domain to parse the preset audio routing configuration file and obtain the audio routing configuration line; wherein, the audio routing configuration line includes at least the following configuration information: the session identifier of the target virtual device node, the second sound card type, the first device node identifier, and the second device node identifier; The device node selection module is used to select the corresponding target virtual device node under the first sound card according to the session identifier, and to select the associated and paired first device node and second device node under the corresponding second sound card according to the second sound card type, the first device node identifier and the second device node identifier. The audio route generation module is used to connect the selected target virtual device node, the first device node, and the second device node according to each audio route configuration line to generate the corresponding preset audio route relationship. The thread creation module is used to create a corresponding number of audio processing threads in the second operating system domain according to the number of lines in the audio routing configuration.

10. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the cross-domain audio processing method according to any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the cross-domain audio processing method according to any one of claims 1-8.

Images