Dynamic communication method and device for multi-core audio task, audio system and equipment

Abstract

The invention provides a dynamic communication method and device for a multi-core audio task, an audio system and equipment. The dynamic communication method for the multi-core audio task comprises the steps that a configuration file of an audio task chain is obtained, the configuration file is generated by configuring the audio task chain, and the audio task chain comprises information associated with a processor core, an audio algorithm and the association relation between the processor core and the audio algorithm; analyzing and loading the configuration file to obtain a dynamic communication mechanism of the audio task chain, the dynamic communication mechanism involving communication among a plurality of processor cores executing audio tasks; and executing a processing action on the input audio signal according to the audio task chain based on the dynamic communication mechanism to obtain an audio processing result. According to the scheme, the audio processing quality when a plurality of processor cores work cooperatively can be improved.

Description

TECHNICAL FIELD

[0001] The present disclosure belongs to the technical field of audio processing, and relates to a dynamic communication method and device for multi-core audio tasks, an audio system and an electronic device. BACKGROUND

[0002] When an audio processing pipeline is executed under a multi-core processor architecture, the cooperation between cores relies on an efficient and dynamic data exchange mechanism, i.e., dynamic communication of multi-core audio tasks. This mechanism involves real-time data transfer, synchronization and control signal exchange between different processing units for the purpose of real-time and high-quality processing of an audio data stream. SUMMARY

[0003] The present disclosure provides a dynamic communication method and device for multi-core audio tasks, an audio system and an electronic device, for improving the quality of audio processing.

[0004] In a first aspect, the present disclosure provides a dynamic communication method for multi-core audio tasks. The method comprises: obtaining a configuration file of an audio task chain, the configuration file being generated by configuring the audio task chain, the audio task chain containing information associated with a processor core, an audio algorithm and an association relationship between the processor core and the audio algorithm; parsing and loading the configuration file to obtain a dynamic communication mechanism of the audio task chain, the dynamic communication mechanism involving communication between a plurality of processor cores that execute audio tasks; and performing a processing action on an input audio signal according to the audio task chain based on the dynamic communication mechanism to obtain an audio processing result.

[0005] In an implementation form of the first aspect, performing the processing action on the input audio signal according to the audio task chain based on the dynamic communication mechanism comprises: performing a corresponding audio algorithm on the input audio signal by the plurality of processor cores according to the association relationship through inter-core communication cooperation.

[0006] In an implementation form of the first aspect, performing the processing action on the input audio signal according to the audio task chain based on the dynamic communication mechanism comprises: performing an audio task using a homogeneous multi-core audio algorithm according to the audio task chain based on a homogeneous multi-core dynamic communication mechanism; performing an audio task using a heterogeneous multi-core audio algorithm according to the audio task chain based on a heterogeneous multi-core dynamic communication mechanism; performing a heavy-load audio task using a multi-core audio algorithm according to the audio task chain based on a heavy-load audio task dynamic communication mechanism; and / or performing a composite audio task using a multi-core audio algorithm according to the audio task chain based on a composite audio task dynamic communication mechanism.

[0007] In an implementation form of the first aspect, performing a processing action on an input audio signal according to the audio task chain based on the dynamic communication mechanism comprises: obtaining dynamic communication information of the audio task chain and audio algorithms corresponding to each processor core according to the configuration file; configuring dynamic communication resources of an audio task according to the dynamic communication information, and constructing an audio task chain of the audio task according to the dynamic communication information and the audio algorithms; and performing priority sorting processing on the audio task chain according to the dynamic communication information, to execute corresponding audio algorithms on the input audio signal.

[0008] In an implementation form of the first aspect, performing an audio task according to the audio task chain based on the dynamic communication mechanism of the homogeneous multi-core comprises: obtaining node information, topology information and dynamic communication information of each node of the homogeneous multi-core audio task; configuring dynamic communication resources of the audio task based on the node information, the topology information and the dynamic communication information; routing the dynamic communication information to a slave processor core by a master processor core; filtering the dynamic communication information according to an identity number by the slave processor core, and performing sorting processing on the audio task chain according to a priority of the identity number, to execute corresponding audio algorithms.

[0009] In an implementation form of the first aspect, performing an audio task according to the audio task chain based on the dynamic communication mechanism of the homogeneous multi-core comprises: obtaining node information, topology information and dynamic communication information of each node of the homogeneous multi-core audio task; configuring dynamic communication resources of the audio task based on the node information, the topology information and the dynamic communication information; routing the dynamic communication information to a slave processor core by a master processor core; filtering the dynamic communication information according to an identity number by the slave processor core, and performing sorting processing on the audio task chain according to a priority of the identity number, to execute corresponding audio algorithms.

[0010] In an implementation form of the first aspect, performing a heavy-load audio task according to the audio task chain based on the dynamic communication mechanism of the heavy-load audio task comprises: obtaining a processing time of the heavy-load audio task according to algorithm parameters and audio parameters; and performing splicing and splitting processing on the heavy-load audio task according to the audio task chain based on the processing time, to obtain processed audio data.

[0011] In one implementation of the first aspect, performing splicing and splitting processing on the heavy-load audio task based on the audio task chain according to the processing time includes: the main processor core distributing the heavy-load audio task to the slave processor core; the main processor core controlling the slave processor core through the identification number of the audio task chain; the slave processor core splicing the audio data in the heavy-load audio task to obtain spliced ​​audio data; the slave processor core executing an audio algorithm on the spliced ​​audio data core to obtain processed audio data; and the slave processor core splitting the processed audio data into multiple audio data blocks and returning the multiple audio data blocks to the main processor core.

[0012] In one implementation of the first aspect, executing a composite audio task using a multi-core audio algorithm based on the composite audio task dynamic communication mechanism according to the audio task chain includes: obtaining the processing time of the composite audio task according to algorithm parameters and audio parameters; and allocating processing to the composite audio task according to the audio task chain based on the processing time to obtain processed audio data.

[0013] In one implementation of the first aspect, allocating processing of the composite audio task based on the processing time according to the audio task chain includes: dividing the composite audio task into a heavy-load audio task and a light-load audio task by the main processor core; distributing the heavy-load audio task to a slave processor core by the main processor core, and controlling the slave processor core by the identification number of the audio task chain; executing a first audio algorithm by the main processor core to process the light-load audio task to obtain a light-load audio processing result; executing a second audio algorithm by the slave processor core to process the heavy-load audio task to obtain a heavy-load audio processing result; sending the heavy-load audio processing result from the slave processor core to the main processor core; and fusing the light-load audio processing result and the heavy-load audio processing result by the main processor core to obtain the processed audio data.

[0014] Secondly, this disclosure provides a dynamic communication device for multi-core audio tasks. The device includes: a configuration file acquisition module configured to acquire a configuration file for an audio task chain, the configuration file being generated by configuring the audio task chain, the audio task chain containing information associated with processor cores, audio algorithms, and the relationship between the processor cores and the audio algorithms; a dynamic communication mechanism acquisition module configured to parse and load the configuration file to acquire a dynamic communication mechanism for the audio task chain, the dynamic communication mechanism involving communication between multiple processor cores executing audio tasks; and an audio processing module configured to perform processing actions on the audio signal according to the audio task chain based on the dynamic communication mechanism to acquire audio processing results.

[0015] Thirdly, this disclosure provides an audio system. The audio system includes: a multimedia terminal configured to configure an audio task chain using an audio algorithm deployment manager to generate a configuration file for the audio task chain; and an audio device configured to: receive the configuration file of the audio task chain from the multimedia terminal, the audio task chain containing information associated with processor cores, audio algorithms, and the association between the processor cores and the audio algorithms; parse and load the configuration file to obtain a dynamic communication mechanism for the audio task chain, the dynamic communication mechanism involving communication between multiple processor cores executing audio tasks; and perform processing actions on input audio signals according to the audio task chain based on the dynamic communication mechanism to obtain audio processing results.

[0016] Fourthly, this disclosure provides an electronic device. The electronic device includes: a memory configured to store audio-related information; and at least one processor electrically coupled to the memory and configured to perform the dynamic communication method for the multi-core audio task described above.

[0017] According to embodiments of this disclosure, the audio algorithm chain in a multi-core audio task constructs a dynamic communication mechanism when multiple processor cores in an audio device execute audio tasks. It dynamically configures communication for audio tasks with homogeneous multi-core and multi-node architectures and heterogeneous multi-core and multi-node architectures, and optimizes the communication mechanism to solve the communication latency problem.

[0018] According to embodiments of this disclosure, the processing time of audio tasks is evaluated using algorithm parameters and audio parameters. Heavy-load audio tasks are distributed to other processor cores for task partitioning and processing. Heterogeneous task converters and unified message formats resolve the issues of heterogeneous communication protocols and cross-hardware core task dependencies. Furthermore, task optimization is achieved by splicing and splitting heavy-load audio tasks, resolving audio timing and scalability issues, and improving the real-time performance and reliability of audio task execution. Attached Figure Description

[0019] Figure 1 The diagram shows a scenario of audio device-multimedia terminal interaction as described in an embodiment of this disclosure.

[0020] Figure 2 The diagram shows a process schematic of the dynamic communication method for multi-core audio tasks as described in an embodiment of this disclosure.

[0021] Figure 3 The diagram illustrates the process of using a homogeneous multi-core audio algorithm to perform audio tasks as described in an embodiment of this disclosure.

[0022] Figure 4 The diagram illustrates the process of using a heterogeneous multi-core audio algorithm to perform audio tasks as described in an embodiment of this disclosure.

[0023] Figure 5 This diagram illustrates the process of performing splicing and splitting processing on a heavy-load audio task as described in an embodiment of this disclosure.

[0024] Figure 6 This diagram illustrates the process of assigning and processing composite audio tasks as described in an embodiment of this disclosure.

[0025] Figure 7 The diagram shown is a structural schematic of a dynamic communication device for multi-core audio tasks according to an embodiment of this disclosure.

[0026] Figure 8 The diagram shown is a structural schematic of the electronic device described in an embodiment of this disclosure. Detailed Implementation

[0027] The following specific examples illustrate the implementation of this disclosure. Those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. This disclosure can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0028] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. Therefore, the drawings only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0029] When executing an audio processing pipeline in a multi-core processor architecture, the collaborative work between cores relies on an efficient and dynamic data exchange mechanism, namely, dynamic communication between multi-core audio tasks. This mechanism involves real-time data transfer, synchronization, and control signal exchange between different processing units to achieve real-time, high-quality processing of audio data streams.

[0030] When multiple algorithm nodes (IO nodes, audio algorithm, mixing nodes, sound effect nodes) in the audio algorithm chain of a multi-core chip communicate for audio tasks, the following problems are usually encountered:

[0031] 1) Load balancing: Multiple cores and nodes share limited system resources, which may lead to access conflicts or system overload.

[0032] 2) Communication latency: In multi-core, multi-node systems, communication latency between nodes can affect the real-time performance of the system. Low latency is especially critical in audio tasks.

[0033] 3) Audio timing issues: Since different nodes in a multi-core system run on different processing cores, it is necessary to ensure the synchronization of each node; otherwise, audio signal distortion, loss, or misalignment may occur.

[0034] 4) Data consistency: Data consistency issues may lead to incorrect data transmission, which in turn affects the quality of audio processing.

[0035] 5) Scalability: Maintaining system stability and performance is a challenge as the number of nodes increases.

[0036] Some technical solutions combine chip-level audio segmentation, AI-accelerated evaluation, task resource allocation strategies, and task scheduling strategy tables to demonstrate the chip's ability to accelerate audio model processing and extract multimodal features at the underlying hardware level. However, their technology selection is clearly constrained by the requirement that "the underlying audio AI operator has the highest operating efficiency."

[0037] To address at least the aforementioned issues, this disclosure provides a dynamic communication scheme for multi-core audio tasks, as illustrated in the following embodiments. This scheme establishes a dynamic communication mechanism for multiple processor cores executing audio tasks in an audio device, dynamically configuring communication for both homogeneous and heterogeneous multi-core multi-node audio tasks, and optimizing the communication mechanism to resolve communication latency issues.

[0038] According to embodiments of this disclosure, the processing time of audio tasks is evaluated using algorithm parameters and audio parameters. Heavy-load audio tasks are distributed to other processor cores for task partitioning and processing. Heterogeneous task converters and unified message formats resolve the issues of heterogeneous communication protocols and cross-hardware core task dependencies. Furthermore, task optimization is achieved by splicing and splitting heavy-load audio tasks, resolving audio timing and scalability issues, and improving the real-time performance and reliability of audio task execution.

[0039] The technical solutions in the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0040] The dynamic communication method for multi-core audio tasks described in this disclosure can be applied to audio device-multimedia terminal interaction scenarios. Figure 1 This is a schematic diagram illustrating the audio device-multimedia terminal interaction scenario described in an embodiment of this disclosure. Figure 1 As shown, the audio device-multimedia terminal interaction system 100 includes an audio device 101 and a multimedia terminal 102. The multimedia terminal 102 is equipped with an audio algorithm layout manager, and the audio device 101 and the multimedia terminal 102 can communicate with each other, and the communication method is not limited to wired or wireless.

[0041] Figure 2 This diagram illustrates the process of a dynamic communication method for multi-core audio tasks according to an embodiment of this disclosure. Figure 2 As shown, the dynamic communication method for the multi-core audio task includes the following steps S100 to S102.

[0042] Step S100: Obtain the configuration file for the audio task chain. The configuration file is generated by configuring the audio task chain through the audio algorithm deployment manager. The audio task chain contains information related to the processor core, the audio algorithm, and the association between the processor core and the audio algorithm.

[0043] In some embodiments, the audio algorithm deployment manager is located on a PC (Personal Computer) and is responsible for generating the audio algorithm chain configuration required to complete the audio task. The audio algorithm chain stores the audio algorithms, algorithm parameters, and algorithm connection relationships required to complete the audio task in the form of a graph. The audio algorithm chain is stored in a configuration file format, such as JSON or XML. The PC-side audio algorithm deployment manager distributes the audio algorithm chain to the audio device, which reads and parses the audio algorithm chain. The audio device constructs the runtime environment for the audio algorithm chain, driving the audio data to be processed within the audio algorithm chain to achieve the predetermined audio processing effects.

[0044] Step S101: The configuration file is parsed and loaded to obtain the dynamic communication mechanism of the audio task chain. The dynamic communication mechanism of the audio task chain is configured through the audio algorithm scheduling manager.

[0045] In some embodiments, the audio device parses the configuration file and constructs a communication and collaboration environment between the processor cores of the audio algorithms based on the audio algorithm chain. The dynamic communication mechanism between processor cores involves communication between multiple processor cores executing audio tasks, including homogeneous multi-core dynamic communication mechanisms, heterogeneous multi-core dynamic communication mechanisms, dynamic communication mechanisms for heavy-load audio tasks, and dynamic communication mechanisms for complex audio tasks.

[0046] For example, homogeneous multi-core is such as multiple DSP (Digital Signal Processor) cores, and heterogeneous multi-core is such as DSP core + NPU (Neural Network Processor) core.

[0047] Step S102: Based on the dynamic communication mechanism, perform processing actions on the input audio signal according to the audio task chain to obtain the audio processing result.

[0048] In one embodiment of this disclosure, performing processing actions on the input audio signal according to the audio task chain based on the dynamic communication mechanism includes: multiple processor cores cooperating through inter-core communication to execute corresponding audio algorithms on the input audio signal according to the association relationship.

[0049] In one embodiment of this disclosure, performing processing actions on the input audio signal according to the audio task chain based on the dynamic communication mechanism includes: executing audio tasks using a homogeneous multi-core audio algorithm according to the audio task chain based on the homogeneous multi-core dynamic communication mechanism; executing audio tasks using a heterogeneous multi-core audio algorithm according to the audio task chain based on the heterogeneous multi-core dynamic communication mechanism; executing heavy-load audio tasks using a multi-core audio algorithm according to the audio task chain based on the heavy-load audio task dynamic communication mechanism; and / or executing composite audio tasks using a multi-core audio algorithm according to the audio task chain based on the composite audio task dynamic communication mechanism.

[0050] In one embodiment of this disclosure, performing processing actions on the input audio signal according to the audio task chain based on the dynamic communication mechanism includes the following steps S200 to S202.

[0051] Step S200: Obtain the dynamic communication information of the audio task chain and the audio algorithm corresponding to each processor core according to the configuration file.

[0052] Step S201: Configure the dynamic communication resources of the audio task according to the dynamic communication information, and construct the audio task chain of the audio task according to the dynamic communication information and the audio algorithm.

[0053] Step S202: Prioritize the audio task chain according to the dynamic communication information to execute the corresponding audio algorithm on the input audio signal.

[0054] Figure 3 This diagram illustrates the process of using a homogeneous multi-core audio algorithm to execute audio tasks in an embodiment of this disclosure. Figure 3 As shown, the audio task is executed using a homogeneous multi-core audio algorithm based on the homogeneous multi-core dynamic communication mechanism according to the audio task chain, including the following steps S300 to S303.

[0055] Step S300: Obtain node information, topology information, and dynamic communication information for each node of the homogeneous multi-core audio task. Node information includes node type, node parameters, and computing cores. Topology information includes input interfaces and input-output relationships. Dynamic communication information includes communication ID and communication mapping information. The communication ID is an identification number used to represent priority encoding, with task priority embedded in the high bits of the ID; for example, ID = 0x8XXX indicates high priority, and 0x0XXX indicates low priority. Communication mapping information is represented as node ID, belonging core, target core, priority, and mailbox (communication mechanism) channel number stored in XML / binary / JSON format.

[0056] Specifically, the audio algorithm deployment manager on the PC configures the node information and topology information of the audio tasks, and the audio device reads the communication ID and communication mapping information of the audio tasks from the configuration file. At the same time, the audio algorithm deployment manager configures the communication ID and communication mapping information of the audio task nodes, and assigns a globally unique ID to the audio task nodes that communicate between cores. For example, core 0 → core 1 is node 0 → ID = 0, core 0 → core 1 is node 1 → ID = 1, and core 1 → core 2 is node 0 → ID = 2.

[0057] Step S301: Configure the dynamic communication resources of the audio task based on the node information, the topology information, and the dynamic communication information.

[0058] Specifically, the audio device reads the communication ID and communication mapping information of the audio task, and configures the communication information of the homogeneous multi-core DSP audio task, parses the node ID and mailbox channel. The audio device configures the dynamic communication resources of the audio task: binds the mailbox channel, and pre-allocates an independent buffer for each ID corresponding to the channel.

[0059] In step S302, the main processor core routes the dynamic communication information to the slave processor core.

[0060] For example, the main processor core is DSP core A, and the slave processor core is the target DSP core. DSP core A encapsulates the audio data of the audio task into the format {target ID, priority, audio payload}, and automatically sends it to the target DSP core through the mailbox channel associated with the ID. For example, DSP0->DSP1 (ID=0 noise reduction) means that DSP0 automatically sends the audio data to DSP1 through ID=0 for noise reduction processing.

[0061] In step S303, the processor core filters the dynamic communication information according to the identity number and sorts the audio task chain according to the priority of the identity number to execute the corresponding audio algorithm.

[0062] For example, the target DSP core performs multi-core communication filtering, meaning DSP1 only listens to messages with ID=0 and ID=1. The slave processor core sorts messages according to the priority embedded in the ID. That is, the target DSP core uses priority rules to sort multiple task types, for example, giving priority to messages with high priority ID=0x8XXX.

[0063] By statically pre-configuring and binding IDs to channels, runtime negotiation overhead is avoided. Simultaneously, mailbox hardware filtering and scheduling based on the ID number enables simplified routing and reduces software parsing latency.

[0064] In some embodiments, assuming the audio chip has homogeneous DSP multi-core (DSP0, DSP1, and DSP2), the implementation process of dynamic communication of audio tasks based on homogeneous multi-core multi-node is represented as follows: the audio algorithm deployment manager configures audio task node information and topology information; the audio algorithm deployment manager configures the communication ID and communication mapping information of the audio task nodes; the audio device reads the communication ID and communication mapping information of the audio task; the audio device configures the dynamic communication resources of the audio task; DSP core A constructs dynamic communication messages and routes them to the target DSP core; the target DSP core filters dynamic communication messages according to the ID; the target DSP core sorts and processes the messages according to the priority embedded in the ID.

[0065] Figure 4 This diagram illustrates the process of using a heterogeneous multi-core audio algorithm to execute audio tasks in an embodiment of this disclosure. Figure 4 As shown, the process of executing audio tasks using a heterogeneous multi-core audio algorithm based on the heterogeneous multi-core dynamic communication mechanism according to the audio task chain includes the following steps S400 to S404.

[0066] Step S400: Obtain node information, topology information, and dynamic communication information of each node in the heterogeneous multi-core audio task.

[0067] Specifically, the audio algorithm deployment manager on the PC configures the node information and topology information of the audio tasks, and the audio device reads the communication ID and communication mapping information of the audio tasks in the configuration file.

[0068] Step S401: Initialize and configure the multi-core dynamic communication manager. When a heterogeneous multi-core audio task starts, it registers its own type and associated system resources with the multi-core dynamic communication manager.

[0069] Step S402: Configure the dynamic communication resources of the audio task based on the node information, the topology information, and the dynamic communication information. The audio device reads and configures the communication information of the heterogeneous multi-core audio task, and parses the node ID and mailbox channel.

[0070] Step S403: Construct a heterogeneous multi-core audio task chain using the dynamic communication information, and drive the audio task chain to perform audio task processing.

[0071] For example, the audio task chain of a heterogeneous multi-core CPU can be constructed as follows: CPU0 (audio acquisition) -> NPU1 (speech recognition) -> DSP2 (echo cancellation) -> CPU0 (result output).

[0072] Step S404: Prioritize the audio task chain using the dynamic communication information to execute the corresponding audio algorithm.

[0073] For example, CPU0 sends the original audio to NPU1 with ID=1. After NPU1 completes the recognition, it sends the text to DSP2 with ID=2. After processing, DSP2 returns the result to CPU0 with ID=3.

[0074] Furthermore, dynamic communication information is used to prioritize audio task chains. The dynamic communication manager arbitrates the transmission order based on the priority field in the ID, for example, hard real-time tasks of the DSP take precedence over batch computations of the NPU.

[0075] The dynamic communication of heterogeneous multi-core processors solves the problems of heterogeneous communication protocols and memory model conflicts through a heterogeneous task converter, and resolves the dependencies between hardware core tasks (CPU preprocessing data → NPU inference → DSP post-processing) by unifying message formats.

[0076] In some embodiments, assuming the audio chip has heterogeneous DSP multi-core (CPUx2, DSPx3, NPUx2), the implementation process of dynamic communication of audio tasks based on heterogeneous multi-core multi-node is represented as follows: the audio algorithm deployment manager configures the dynamic communication information of heterogeneous multi-core multi-node; the audio device configures and initializes the multi-core dynamic communication manager; the audio device configures the dynamic communication resources of audio tasks; the audio device uses the dynamic communication information to construct the audio task chain of heterogeneous multi-core; the audio device uses the dynamic communication information to drive the audio task chain to process audio tasks; and the audio device uses the dynamic communication information to schedule the audio task chain with mixed priorities.

[0077] In one embodiment of this disclosure, the execution of a heavy-load audio task using a multi-core audio algorithm based on the audio task chain according to the heavy-load audio task dynamic communication mechanism includes: obtaining the processing time of the heavy-load audio task according to algorithm parameters and audio parameters; and based on the processing time, the audio task chain performs splicing and splitting processing on the heavy-load audio task to obtain processed audio data.

[0078] Figure 5 This diagram illustrates the process of splicing and splitting heavy audio tasks in an audio task chain according to an embodiment of this disclosure. Figure 5 As shown, the process of performing splicing and splitting of the heavy-load audio task based on the processing time and the audio task chain includes the following steps S500 to S503.

[0079] In step S500, the main processor core distributes the heavy-load audio tasks to the slave processor cores. The main processor core controls the slave processor cores using the identification number of the audio task chain.

[0080] Step S501: The processor core splices the audio data in the heavy-load audio task to obtain spliced ​​audio data.

[0081] Step S502: The processor core executes an audio algorithm on the spliced ​​audio data core to obtain processed audio data.

[0082] Step S503: The processor core splits the processed audio data into multiple audio data blocks and returns the multiple audio data blocks to the main processor core.

[0083] In some embodiments, assuming the audio algorithm chain configured in the audio device is DSP0 (32 Bytes) -> DSP1 (concatenating 768 Bytes) -> DSP1 (audio algorithm B) -> DSP1 (splitting 32 Bytes) -> DSP0 (32 Bytes), then based on the above audio algorithm chain, the concatenation and splitting process for multi-core, multi-node audio data is represented as follows: DSP0 evaluates the processing time of the audio task through algorithm parameters and audio parameters; DSP0 distributes the heavy-load audio task to DSP1; DSP1 concatenates the audio data; DSP1 concatenates multiple 32-Byte audio data into 768-Byte continuous audio data; DSP1 executes the audio processing algorithm; DSP1 performs complex operations such as filtering, gain adjustment, and echo cancellation on the heavy-load audio processing task of DSP0; DSP1 splits the audio data; after executing the audio processing algorithm, DSP1 obtains 768-Byte audio processing result data, splits the 768-Byte audio processing result data into multiple 32-Byte data blocks, and transmits them back to DSP0 one by one.

[0084] In one embodiment of this disclosure, executing a composite audio task using a multi-core audio algorithm based on the composite audio task dynamic communication mechanism according to the audio task chain includes: obtaining the processing time of the composite audio task according to algorithm parameters and audio parameters; and allocating processing of the composite audio task according to the audio task chain based on the processing time to obtain processed audio data.

[0085] Figure 6 This diagram illustrates the process of allocating and processing composite audio tasks using an audio task chain in one embodiment of this disclosure. Figure 6 As shown, the process of allocating and processing the composite audio task according to the audio task chain based on the processing time includes the following steps S600 to S605.

[0086] In step S600, the main processor core divides the composite audio task into a heavy-load audio task and a light-load audio task.

[0087] In step S601, the main processor core distributes the heavy-load audio tasks to the slave processor cores. The main processor core controls the slave processor cores using the identification number of the audio task chain.

[0088] In step S602, the main processor executes the first audio algorithm to process the light-load audio task in order to obtain the light-load audio processing result.

[0089] Step S603: The second audio algorithm is executed from the processor core to process the heavy-load audio task in order to obtain the heavy-load audio processing result.

[0090] Step S604: The slave processor core sends the heavy-load audio processing result to the main processor core.

[0091] In step S605, the main processor core fuses the light-load audio processing result and the heavy-load audio processing result to obtain the processed audio data.

[0092] In some embodiments, assuming the audio algorithm chain configured for the audio device is as follows, the processing procedure for multi-core, multi-node audio long-frame composite audio tasks based on this audio algorithm chain is represented as follows:

[0093]

[0094] DSP0 evaluates the processing time of the audio task based on the algorithm parameters and audio parameters; DSP0 distributes the heavy-load audio task to DSP1; DSP0 executes audio algorithm A to complete the light-load audio task and obtains the light-load audio processing result; DSP1 executes audio algorithm B to complete the heavy-load audio task and obtains the heavy-load audio processing result; DSP1 sends the heavy-load audio processing result back to DSP0; DSP0 merges the light-load audio processing result and the heavy-load audio processing result to obtain the final audio processing result.

[0095] The scope of protection for the dynamic communication method for multi-core audio tasks described in this disclosure is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this disclosure is included within the scope of protection of this disclosure.

[0096] This disclosure also provides a dynamic communication device for multi-core audio tasks. The dynamic communication device for multi-core audio tasks can implement the dynamic communication method for multi-core audio tasks described in this disclosure. However, the implementation device for the dynamic communication method for multi-core audio tasks described in this disclosure includes, but is not limited to, the structure of the dynamic communication device for multi-core audio tasks listed in this embodiment. Any structural modifications and substitutions of the prior art made based on the principles of this disclosure are included within the protection scope of this disclosure.

[0097] Figure 7 The diagram shown is a structural schematic of a dynamic communication device for multi-core audio tasks according to an embodiment of this disclosure. Figure 7 As shown, the dynamic communication device 1 for the multi-core audio task includes: a configuration file acquisition module 11, a dynamic communication mechanism acquisition module 12, and an audio processing module 13.

[0098] The configuration file acquisition module 11 is configured to acquire the configuration file of the audio task chain, which is generated by configuring the audio task chain. The audio task chain contains information related to the processor core, the audio algorithm, and the association between the processor core and the audio algorithm.

[0099] The dynamic communication mechanism acquisition module 12 is configured to parse and load the configuration file to obtain the dynamic communication mechanism of the audio task chain, the dynamic communication mechanism involving communication between multiple processor cores executing audio tasks; and

[0100] The audio processing module 13 is configured to perform processing actions on the input audio signal according to the audio task chain based on the dynamic communication mechanism, so as to obtain the audio processing result.

[0101] It should be noted that, Figure 7 The modules in the dynamic communication device 1 for the multi-core audio task shown are connected to... Figure 2 The steps in the dynamic communication method for the multi-core audio task correspond one-to-one, and will not be elaborated here.

[0102] This disclosure also provides a display system. The display system includes a multimedia terminal and an audio device. The multimedia terminal is configured to configure an audio task chain using an audio algorithm deployment manager to generate a configuration file for the audio task chain. The audio device is configured to receive the configuration file of the audio task chain from the multimedia terminal. The audio task chain contains information associated with processor cores, audio algorithms, and the association between the processor cores and the audio algorithms. The audio device is configured to parse and load the configuration file to obtain a dynamic communication mechanism for the audio task chain, the dynamic communication mechanism involving communication between multiple processor cores executing audio tasks. Furthermore, the audio device is configured to perform processing actions on input audio signals according to the audio task chain based on the dynamic communication mechanism to obtain audio processing results.

[0103] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, or methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or modules or units may be electrical, mechanical, or other forms.

[0104] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules / units can be selected to achieve the objectives of the embodiments of this disclosure, depending on actual needs. For example, the functional modules / units in the various embodiments of this disclosure may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0105] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0106] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the dynamic communication method for multi-core audio tasks provided in this disclosure. Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing a processor. The program can be stored in a computer-readable storage medium, which is a non-transitory medium, such as random access memory, read-only memory, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof. The storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0107] This disclosure also provides an electronic device. Figure 8 The diagram shown is a structural schematic of an electronic device 2 according to an embodiment of this disclosure. Figure 8 As shown, in this embodiment, the electronic device 2 includes a memory 21 and a processor 22.

[0108] The memory 21 is configured to store audio-related information. In some possible implementations, the memory 21 may include various media capable of storing program code, such as ROM, RAM, magnetic disk, USB flash drive, memory card, or optical disk.

[0109] In this embodiment of the disclosure, memory 21 may include a computer system readable medium in the form of volatile memory, such as RAM and / or cache memory. Electronic device 2 may further include other removable / non-removable, volatile / non-volatile computer system storage media. Memory 21 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present disclosure.

[0110] At least one processor 22 is electrically coupled to the memory 21 for executing a computer program stored in the memory 21 to enable the electronic device 2 to perform a dynamic communication method for multi-core audio tasks.

[0111] For example, processor 22 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. In other embodiments, processor 22 may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0112] In some implementations, the electronic device 2 provided in this disclosure embodiment may further include a display 23. The display 23 is communicatively connected to the memory 21 and the processor 22, and is used to display a graphical user interface (GUI) related to the dynamic communication method of the multi-core audio task.

[0113] In this embodiment, the display 23 may include a display screen (display panel). In some implementations, the display panel may be configured using a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like. Alternatively, the display 23 may also be a touch panel (touchscreen, touch screen), which may include a display screen and a touch-sensitive surface. When the touch-sensitive surface detects a touch operation on or near it, it transmits the information to the processor 22 to determine the type of touch event. Subsequently, the processor 22 provides corresponding visual output on the display device based on the type of touch event.

[0114] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0115] The above embodiments are merely illustrative of the principles and effects of this disclosure and are not intended to limit this disclosure. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this disclosure. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this disclosure should still be covered by the claims of this disclosure.

Claims

1. A dynamic communication method for multi-core audio tasks, characterized in that, include: Obtain the configuration file for the audio task chain, which is generated by configuring the audio task chain. The audio task chain contains information related to the processor core, the audio algorithm, and the relationship between the processor core and the audio algorithm. The configuration file is parsed and loaded to obtain the dynamic communication mechanism of the audio task chain, which involves communication between multiple processor cores executing audio tasks; as well as Based on the dynamic communication mechanism, the input audio signal is processed according to the audio task chain to obtain the audio processing result.

2. The data synchronization method for multi-core audio tasks according to claim 1, characterized in that, Based on the dynamic communication mechanism, the processing actions performed on the input audio signal according to the audio task chain include: Multiple processor cores collaborate through inter-core communication to execute corresponding audio algorithms on the input audio signal based on the aforementioned correlation.

3. The dynamic communication method for multi-core audio tasks according to claim 1, characterized in that, Based on the dynamic communication mechanism, the processing actions performed on the input audio signal according to the audio task chain include: Based on the homogeneous multi-core dynamic communication mechanism, the audio task is executed using the homogeneous multi-core audio algorithm according to the audio task chain; Based on the heterogeneous multi-core dynamic communication mechanism, the audio task is executed using a heterogeneous multi-core audio algorithm according to the audio task chain; Based on the dynamic communication mechanism for heavy-load audio tasks, a multi-core audio algorithm is used to execute heavy-load audio tasks according to the audio task chain; and / or Based on the dynamic communication mechanism for composite audio tasks, a multi-core audio algorithm is used to execute composite audio tasks according to the audio task chain.

4. The dynamic communication method for multi-core audio tasks according to claim 1, characterized in that, Based on the dynamic communication mechanism, the processing actions performed on the input audio signal according to the audio task chain include: The dynamic communication information of the audio task chain and the audio algorithm corresponding to each processor core are obtained according to the configuration file. Configure dynamic communication resources for the audio task based on the dynamic communication information, and construct the audio task chain for the audio task based on the dynamic communication information and the audio algorithm; and The audio task chain is prioritized based on the dynamic communication information to execute the corresponding audio algorithm on the input audio signal.

5. The dynamic communication method for multi-core audio tasks according to claim 3, characterized in that, Based on the homogeneous multi-core dynamic communication mechanism, the audio task is executed using a homogeneous multi-core audio algorithm according to the audio task chain, including: Obtain node information, topology information, and dynamic communication information of each node in a homogeneous multi-core audio task; Configure the dynamic communication resources of the audio task based on the node information, the topology information, and the dynamic communication information; The main processor core routes the dynamic communication information to the slave processor core; and The processor core filters the dynamic communication information based on the identity number and sorts the audio task chain according to the priority of the identity number in order to execute the corresponding audio algorithm.

6. The dynamic communication method for multi-core audio tasks according to claim 3, characterized in that, Based on the heterogeneous multi-core dynamic communication mechanism, the audio task chain is executed using a heterogeneous multi-core audio algorithm, including: Obtain node information, topology information, and dynamic communication information of each node in the heterogeneous multi-core audio task; Initialize and configure the multi-core dynamic communication manager; Configure the dynamic communication resources of the audio task based on the node information, the topology information, and the dynamic communication information; The dynamic communication information is used to construct a heterogeneous multi-core audio task chain, and the audio task chain is driven to perform audio task processing; and The dynamic communication information is used to prioritize and schedule the audio task chain in order to execute the corresponding audio algorithm.

7. The dynamic communication method for multi-core audio tasks according to claim 3, characterized in that, The dynamic communication mechanism based on heavy-load audio tasks uses a multi-core audio algorithm to execute heavy-load audio tasks according to the audio task chain, including: The processing time of the heavy-load audio task is obtained based on the algorithm parameters and audio parameters; and Based on the processing time, the heavy-load audio task is spliced ​​and split according to the audio task chain to obtain the processed audio data.

8. The dynamic communication method for multi-core audio tasks according to claim 7, characterized in that, Based on the processing time, the heavy-load audio task is spliced ​​and split according to the audio task chain, including: The main processor core distributes the heavy-load audio tasks to the slave processor cores, and the main processor core controls the slave processor cores through the identity number of the audio task chain; The processor core splices the audio data from the heavy-load audio task to obtain spliced ​​audio data; The processor core executes an audio algorithm on the spliced ​​audio data core to obtain processed audio data; and The processor core splits the processed audio data into multiple audio data blocks and returns the multiple audio data blocks to the main processor core.

9. The dynamic communication method for multi-core audio tasks according to claim 3, characterized in that, Based on the dynamic communication mechanism for composite audio tasks, and according to the audio task chain, a multi-core audio algorithm is used to execute composite audio tasks, including: The processing time of the composite audio task is obtained based on the algorithm parameters and audio parameters; and Based on the processing time, the composite audio task is allocated and processed according to the audio task chain to obtain the processed audio data.

10. The dynamic communication method for multi-core audio tasks according to claim 9, characterized in that, The allocation and processing of the composite audio task based on the processing time and according to the audio task chain includes: The main processor core divides the composite audio task into heavy-load audio tasks and light-load audio tasks; The main processor core distributes the heavy-load audio tasks to the slave processor cores, and the main processor core controls the slave processor cores through the identification number of the audio task chain; The main processor executes a first audio algorithm to process light-load audio tasks in order to obtain light-load audio processing results. The heavy-load audio task is processed by executing a second audio algorithm from the processor core to obtain the heavy-load audio processing result; The slave processor core sends the heavy-load audio processing results to the main processor core; and The main processor core fuses the light-load audio processing results and the heavy-load audio processing results to obtain the processed audio data.

11. A dynamic communication device for multi-core audio tasks, characterized in that, include: The configuration file acquisition module is configured to acquire the configuration file of the audio task chain. The configuration file is generated by configuring the audio task chain, which contains information related to the processor core, the audio algorithm, and the relationship between the processor core and the audio algorithm. A dynamic communication mechanism acquisition module is configured to parse and load the configuration file to obtain the dynamic communication mechanism of the audio task chain, wherein the dynamic communication mechanism involves communication between multiple processor cores executing audio tasks. as well as The audio processing module is configured to perform processing actions on the input audio signal according to the audio task chain based on the dynamic communication mechanism, so as to obtain the audio processing result.

12. An audio system, characterized in that, include: A multimedia terminal is configured to use an audio algorithm layout manager to configure an audio task chain in order to generate a configuration file for the audio task chain. as well as The audio device is configured as follows: The multimedia terminal receives a configuration file for the audio task chain, which contains information related to the processor core, the audio algorithm, and the relationship between the processor core and the audio algorithm. The configuration file is parsed and loaded to obtain the dynamic communication mechanism of the audio task chain, which involves communication between multiple processor cores executing audio tasks; as well as Based on the dynamic communication mechanism, the input audio signal is processed according to the audio task chain to obtain the audio processing result.

13. An electronic device, characterized in that, include: The memory is configured to store audio-related information; as well as At least one processor is electrically coupled to the memory and configured to perform a dynamic communication method for a multi-core audio task according to any one of claims 1-10.

Images