Distributed audio processing method, device and system, electronic equipment and storage medium

Abstract

The invention provides a distributed audio processing method, device and system, electronic equipment and a storage medium. The method comprises the steps that an audio processing task received by a distributed audio processing system is acquired, and data transmission is carried out between at least two processing nodes in the distributed audio processing system based on a target audio transmission protocol; according to a type field in the target audio transmission protocol, identifying a processing type of an audio code stream in the audio processing task; and transmitting the audio code stream to a target processing node corresponding to the processing type in the at least two processing nodes to enable the target processing node to process the audio code stream, and transmitting the obtained processed audio code stream to at least one loudspeaker for playing. The resource utilization rate and performance of the distributed audio processing system can be improved, and resource waste or insufficiency caused by fixed processing node selection is avoided.

Description

Technical Field

[0001] The present application relates to multimedia processing technology, and in particular to a distributed audio processing method, device, system, electronic device and storage medium. Background Art

[0002] More and more modern electronic products such as cars and mobile phones have distributed multi-processor systems to handle a variety of application requirements and achieve the purpose of saving power.

[0003] Among them, multimedia processing is a very popular function in electronic products, and generally provides dedicated processors such as visual processors GPU to support video processing. But for audio processing, it often shares some single instruction stream multiple data stream SIMD instructions of the central processing unit CPU with other multimedia processing to improve parallel processing capabilities.

[0004] However, the emerging spatial audio technology requires much higher processing power than traditional stereo, and the current processor architecture is not able to meet the needs of processing multi-channel spatial audio. For example, in the field of automotive electronics, audio systems are increasingly required to be combined with other systems (such as microphones, displays, seats, lights, etc.). Therefore, regardless of the limitations of processor computing power or input and output interfaces, a distributed solution is required. Summary of the invention

[0005] The present application provides a distributed audio processing method, device, system, electronic device and storage medium to solve the problem that the audio processing architecture in the prior art cannot effectively meet the needs of processing multi-channel spatial audio. It can improve the resource utilization and performance of the distributed audio processing system and avoid resource waste or shortage due to fixed processing node selection.

[0006] In one aspect, the present application provides a distributed audio processing method, the method comprising:

[0007] Acquire an audio processing task received by a distributed audio processing system, wherein data is transmitted between at least two processing nodes in the distributed audio processing system based on a target audio transmission protocol;

[0008] Identify the processing type of the audio code stream in the audio processing task according to the type field in the target audio transmission protocol;

[0009] The audio code stream is transmitted to a target processing node corresponding to the processing type among the at least two processing nodes, so that the audio code stream is processed by the target processing node, and the processed audio code stream is transmitted to at least one speaker for playback.

[0010] In an optional implementation manner, transmitting the audio code stream to a target processing node corresponding to the processing type among the at least two processing nodes includes:

[0011] determining audio processing requirements corresponding to the processing type;

[0012] Determining the target processing node for processing the audio code stream according to the audio processing requirement corresponding to the processing type;

[0013] The audio code stream is transmitted to the target processing node.

[0014] In an optional implementation manner, the protocol data format of the target audio transmission protocol at least includes:

[0015] Multiple data unit fields are used to carry audio code streams of different processing types. Each data unit field contains: type label, unit length, and data segment;

[0016] The data unit quantity field is used to record the number of processing types of the audio code stream in the audio transmission protocol frame.

[0017] In an optional implementation manner, the audio code stream carried in the data unit field includes at least one of the following:

[0018] Encode and compress the audio code stream;

[0019] Multi-channel and object streams with metadata after local decoding;

[0020] After decoding, it is rendered and can be directly output to the original audio stream played by the speaker;

[0021] Sensor data in different forms.

[0022] In an optional implementation manner, the target processing node is used to process the audio code stream to obtain a processed audio code stream, including:

[0023] The master node is used to perform at least one of the following processing on the audio code stream: decoding, rendering, and post-processing to obtain a processed audio code stream.

[0024] In an optional embodiment, using the target processing node to process the audio code stream to obtain a processed audio code stream includes:

[0025] Using a power amplifier slave node among the slave nodes, the audio code stream is processed by at least one of the following: decoding, rendering, and post-processing to obtain a processed audio code stream;

[0026] A multidimensional perception driving slave node among the slave nodes is used to perform driving control processing on the multidimensional perception metadata obtained by audio decoding processing to obtain multidimensional perception driving control data.

[0027] In an optional implementation manner, a multidimensional perception drive slave node among the slave nodes is used to perform drive control processing on the multidimensional perception metadata obtained by audio decoding processing to obtain multidimensional perception drive control data, including at least one of the following:

[0028] Using an ambient light drive slave node to perform drive control processing on the ambient light data related to the spatial audio data to obtain ambient light drive control data;

[0029] Using a vibration drive slave node to perform drive control processing on the vibration data related to the spatial audio data to obtain seat vibration drive control data;

[0030] Using a fragrance drive slave node to perform drive control processing on the fragrance data related to the spatial audio data to obtain fragrance drive control data;

[0031] Using an air-conditioning driving slave node to perform driving control processing on the air-conditioning data related to the spatial audio data to obtain air-conditioning driving control data;

[0032] The glasses driving slave node is used to perform driving control processing on the video data related to the spatial audio data to obtain audio-visual driving control data.

[0033] On the other hand, the present application provides a distributed audio processing system, the system comprising:

[0034] A transmission bus, used for receiving an audio processing task, and identifying a processing type of an audio code stream in the audio processing task according to a type field in a target audio transmission protocol, so as to transmit the audio code stream to a master node or a slave node according to the processing type, wherein the transmission bus performs data transmission based on a target audio transmission protocol, and the target audio transmission protocol supports transmission of different types of audio code streams and transmission modes;

[0035] A master node, used for performing master node processing on the received audio code stream to obtain a processed audio code stream;

[0036] At least one slave node is used to perform slave node processing on the received audio code stream to obtain a processed audio code stream, and transmit the processed audio code stream to at least one speaker for playback, wherein the master node and the slave node are connected using the transmission bus, and transmit the audio code stream and the processed audio code stream.

[0037] In an optional implementation manner, the master node is provided with a first audio subsystem supporting a spatial audio interface, and the first audio subsystem includes:

[0038] A first input layer, used for receiving the audio code streams from different applications and converting the audio code streams into audio data packets in a unified format;

[0039] A mixing layer, used for mixing and sorting the audio data packets transmitted by the first input layer to obtain mixed audio data packets;

[0040] A first output layer, used for distributing the mixed audio data packets output by the mixing layer to the slave node or the terminal node;

[0041] The first configuration layer is used to configure the parameters and functions of the first input layer, the mixing layer and the first output layer according to different playback environments and user requirements.

[0042] In an optional implementation manner, a corresponding second audio subsystem is provided on each of the slave nodes, and the second audio subsystem includes:

[0043] The second input layer is used to receive the audio code stream from the master node or other slave nodes, and convert the audio code stream into an audio data packet in a unified format;

[0044] A processing layer, configured to perform slave node processing on the audio data packet transmitted by the second input layer to obtain a processed audio data packet;

[0045] A second output layer, used for distributing the processed audio data packets output by the processing layer to other slave nodes or the terminal node;

[0046] The second configuration layer is used to configure the parameters and functions of the second input layer, the processing layer and the second output layer according to different playback environments and user requirements.

[0047] On the other hand, the present application provides a distributed audio processing device, the device comprising:

[0048] An acquisition module, configured to acquire an audio processing task received by a distributed audio processing system, wherein data is transmitted between at least two processing nodes in the distributed audio processing system based on a target audio transmission protocol;

[0049] An identification module, used for identifying the processing type of the audio code stream in the audio processing task according to the type field in the target audio transmission protocol;

[0050] A processing module is used to transmit the audio code stream to a target processing node corresponding to the processing type among the at least two processing nodes, so that the audio code stream is processed by the target processing node, and the processed audio code stream is transmitted to at least one speaker for playback.

[0051] On the other hand, the present application provides an electronic device, comprising: a processor, and a memory connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement any of the above methods.

[0052] On the other hand, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement any of the above methods.

[0053] On the other hand, the present application provides a computer program product, including a computer program, which implements any of the above methods when executed by a processor.

[0054] The present application provides a distributed audio processing method, device, system, electronic device and storage medium. The method includes: obtaining an audio processing task received by a distributed audio processing system, wherein at least two processing nodes in the distributed audio processing system perform data transmission based on a target audio transmission protocol; identifying the processing type of an audio code stream in the audio processing task according to a type field in the target audio transmission protocol; transmitting the audio code stream to a target processing node corresponding to the processing type among at least two processing nodes, so that the target processing node is used to process the audio code stream, and transmitting the processed audio code stream to at least one speaker for playback.

[0055] By adopting the embodiments of the present application, a suitable target processing node is selected for processing through the transmission bus according to the type field in the target audio transmission protocol, which can solve the problem that the audio processing architecture in the prior art cannot effectively meet the needs of processing multi-channel spatial audio, improve the resource utilization and performance of the distributed audio processing system, and avoid resource waste or shortage due to fixed processing node selection. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0057] Figure 1 A schematic diagram of the architecture of a distributed audio processing system provided in an embodiment of the present application;

[0058] Figure 2A schematic diagram of a protocol data format of an optional target audio transmission protocol provided in an embodiment of the present application;

[0059] Figure 3 A protocol layer diagram of an optional target audio transmission protocol provided in an embodiment of the present application;

[0060] Figure 4 A schematic diagram of an optional processing module in a master node or a slave node provided in an embodiment of the present application;

[0061] Figure 5 A schematic diagram of an optional distributed 5D spatial audio cockpit system provided in an embodiment of the present application;

[0062] Figure 6 It is a flowchart of a distributed audio processing method provided in an embodiment of the present application;

[0063] Figure 7 It is a flowchart of an optional distributed audio processing method provided in an embodiment of the present application;

[0064] Figure 8 A structural block diagram of a distributed audio processing device provided in an embodiment of the present application;

[0065] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0066] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0067] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0068] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0069] The distributed audio processing method provided by the present application is intended to solve the above technical problems of the prior art. The distributed audio processing method can be applied to Figure 1 A schematic diagram of the architecture of a distributed audio processing system is shown in FIG. Figure 1 As shown, the distributed audio processing system includes:

[0070] The transmission bus 10 is used to receive the audio processing task, and identify the processing type of the audio code stream in the audio processing task according to the type field in the target audio transmission protocol, so as to transmit the audio code stream to the master node or the slave node according to the processing type, wherein the transmission bus performs data transmission based on the target audio transmission protocol, and the target audio transmission protocol supports the transmission of different types of audio code streams and transmission modes;

[0071] A master node 20, used to perform master node processing on the received audio code stream to obtain a processed audio code stream; at least one slave node 30 (when there are multiple slave nodes 30, it can include but is not limited to the following Figure 1 The slave node 30-1, ... slave node 30-n, terminal node) shown is used to perform slave node processing on the received audio code stream to obtain a processed audio code stream, and transmit the processed audio code stream to at least one speaker for playback.

[0072] Optionally, the master node and the slave node are connected via a transmission bus to transmit the audio code stream and the processed audio code stream via the transmission bus.

[0073] Optionally, in an embodiment of the present application, the above-mentioned distributed audio processing system is a vehicle-mounted audio processing system, in which the transmission bus is also used to receive audio processing tasks, and identify and transmit different types of audio code streams according to the type field in the target audio transmission protocol; and, the master node and at least one slave node are connected by a transmission bus, and the transmission bus is used to transmit the audio code stream to be processed and the processed audio code stream.

[0074] In an embodiment of the present application, the master node is a device for receiving, processing and transmitting audio signals, for example, such as spatial audio decoding, sound bed or object multi-channel rendering, post-processing equalization, etc.; the slave node is a device for special audio processing and outputting audio signals, for example, for processing spatial audio decoding, sound bed or object multi-channel rendering, post-processing equalization, multi-dimensional perception driving, etc.

[0075] Optionally, in an embodiment of the present application, audio code streams processed or unprocessed by the master node and sent to the slave node for processing are all audio code streams received by the slave node.

[0076] Still Figure 1As shown, the master node and the slave node are based on the audio transmission protocol, and the audio processing tasks are allocated according to the capabilities between the nodes or the needs of the application. For example, the audio processing business can be allocated according to factors such as computing power, memory space, and hardware power consumption. If a digital signal processing DSP chip has sufficient computing power resources and strong real-time performance, it can perform rendering and sound field calibration, and the workload of the subsequent power amplifier node is very small; otherwise, the sound field calibration work may be handed over to the power amplifier node, and the DSP information is only rendered. Finally, the processed audio stream is output to one or more speakers through the terminal node for playback.

[0077] In one example, after receiving the audio code stream, the master node can transmit the processed or unprocessed audio code stream of the master node to the slave node based on the audio transmission protocol, regardless of whether to process it, so that the slave node processes the received audio code stream to obtain a processed audio code stream, or directly transmits the audio code stream processed by the master node as the above-mentioned processed audio code stream to at least one speaker for playback, and drives seats, ambient lights, air conditioners and other facilities, thereby realizing the output and driving of audio data.

[0078] In one example, for example, for certain audio bitstream types that require more computing resources or special functions, such as spatial audio, three-dimensional stereo, etc., the system can select a suitable target processing node for processing through the transmission bus according to the type field in the target audio transmission protocol, and transmit the processed audio bitstream to the speaker for playback; for another example, for certain audio bitstream types that require fewer computing resources or ordinary functions, such as ordinary stereo, mono, etc., the system can select a suitable target processing node for processing through the transmission bus according to the type field in the target audio transmission protocol, and transmit the processed audio bitstream to the speaker for playback.

[0079] Therefore, by selecting a suitable target processing node for processing according to the type field in the target audio transmission protocol through the transmission bus, the resource utilization and performance of the distributed audio processing system can be improved, and resource waste or shortage caused by fixed processing node selection can be avoided.

[0080] In another example, for different types or versions of processing nodes, the supported audio code stream types and processing functions may be different, which may cause sound quality differences or distortion if fixed processing nodes are used. The system can identify and transmit different types of audio code streams according to the type field in the target audio transmission protocol through the transmission bus, and achieve adaptation and compatibility with different types or versions of processing nodes through negotiation and coordination between the master node and the slave node. The adaptability and compatibility of the distributed audio processing system can be improved, avoiding sound quality differences or distortion caused by different types or versions of processing nodes.

[0081] Optionally, the above-mentioned audio transmission protocol is used to specify the format and transmission method of audio code streams and other related data, realizing the effective distribution and reception of audio data to meet the different deployment requirements of audio processing systems and achieve load balancing between different processor nodes.

[0082] In an optional embodiment, after receiving the audio code stream, the master node generally performs some processing, for example: obtaining the audio code stream and / or 5D data from the system stream by demultiplexing; further parsing the audio code stream, such as parsing out the sound bed, object or scene audio signal; or further partially or completely decoding the parsed audio signal, such as partially or completely decoding the compressed sound bed, object or scene signal, such as decoding it into a PCM signal, a PCM+Metadata signal, or a PCM array signal; or further rendering part or all of the decoded signal to obtain the signal of the target speaker; or further multiplexing the input information of other sensors such as cameras and radars with the audio data of the above different stages, and distributing the audio code stream processed by the master node to other slave nodes in the system, generally dedicated audio processing nodes, for further processing according to a predetermined audio transmission protocol.

[0083] In another embodiment, the embodiment of the present application also supports distributed interaction between nodes: the master node or the slave node can receive user input or data input from various related sensors, and can transmit relevant data between nodes, and these data and audio data are used for fusion rendering; the multi-processor node system that communicates through the transmission protocol is beneficial for load balancing of audio processing computing power, fusion calculation of multiple sensors, collaborative work of multiple drive units, local optimization and upgrading of the system, etc.

[0084] Optionally, the above-mentioned audio data transmission protocol can transmit sensor data in different forms, such as data collected by the Automatic Data Acquisition System (ADAS), speed sensor, etc., which can be transmitted and aggregated into a data stream through the transmission bus and received and processed by each processing node on the bus.

[0085] It should be noted that the above-mentioned sensor data is different from metadata such as seat vibration and ambient light. The former is the data input in real time by the car sensor and is used for subsequent renderer input. The latter may be written during the production of the program content or obtained through AI system analysis.

[0086] As an optional embodiment, in the in-vehicle audio processing system, digital signals generally follow specific communication protocol specifications, such as A2B / I2S and other communication protocols, so the encapsulation and decapsulation of the panoramic sound WANOS protocol between nodes are generally carried out in accordance with the specifications and capacity limits of A2B / I2S. The main processing node generally refers to the signal source, such as in the in-vehicle audio processing system of a car, all the presented content comes from compressed streams and / or input devices such as microphones; the slave nodes are the various modules in the processing process, such as the rendering node after decoding, the 5D signal conversion node after rendering, and so on.

[0087] In an optional embodiment, one or more audio processing nodes within the vehicle audio processing system identify the form of the audio data stream received according to the type field in the audio transmission protocol, and then adopt different audio processing processes, such as spatial audio decoding, sound bed or object multi-channel rendering, post-processing equalization, etc. Depending on the form of the audio stream, it can be full-process processing or partial processing. The principle of allocating processing functions to each node can adopt the principle of proximity. For example, processing related to specific hardware can be completed at the processing node closest to it. The processing capabilities of different nodes can also be allocated based on the principle of minimum energy consumption and optimal system experience.

[0088] In this way, the audio code stream is processed through the coordinated cooperation of multiple audio processing nodes, and finally output to the speaker for playback. At the same time, it can also drive seats, ambient lights, fragrances, air conditioning and other facilities, giving users a 5D cockpit experience.

[0089] An optional implementation method, such as Figure 2 As shown, the protocol data format of the above target audio transmission protocol at least includes:

[0090] Multiple data unit fields (such as Figure 2 As shown, data unit 1 ... data unit s) is used to carry audio code streams of different processing types, and each data unit field includes: type label, unit length, and data segment.

[0091] The data unit quantity field is used to record the number of processing types of the audio code stream in the audio transmission protocol frame.

[0092] Optionally, the target audio transmission protocol may be a panoramic sound WANOS protocol, in which one or more tags may be defined to indicate the processing type of the audio data; as an optional embodiment, the processing type may also be divided into multiple levels, such as "music class" --> "rock class" --> "pure WANOS object class", etc.

[0093] Optionally, the audio data format can be classified according to different angles, such as compression class / uncompression class, sound bed class / object class, etc.; the processing methods corresponding to different classification methods are also different. For example, the compression class needs to be decoded first, but the uncompression class does not need to be decoded; the sound bed class needs to perform the "channel signal->channel signal" rendering method, and the object class needs to perform the "channel signal+object metadata->channel signal" rendering method, etc.; it should be noted that in the embodiments of the present application, multiple classification methods can be combined to obtain more detailed categories.

[0094] In addition, there is an optional embodiment, still as Figure 2 As shown, the above target audio transmission protocol may also include:

[0095] Frame header synchronization field: used by the receiving node to detect the beginning of a transmission protocol frame.

[0096] The frame length field is used to indicate the length of a frame.

[0097] Optional frame check field: can be used to improve the reliability of discovering the frame header. The specific check algorithm can use various mainstream check algorithms such as sum check and CRC check.

[0098] Data unit quantity field: used to indicate how many different types of audio data are in this frame

[0099] Data unit field: used to carry different types of audio data, such as Figure 2 As shown, it may include but is not limited to: a data type label (type label), a data unit length (unit length), a data segment with the data type label (data segment), an optional data unit check field, etc.

[0100] For example, the data segment may carry different types of data, such as a coded and compressed audio stream, a multi-channel object stream with metadata after local decoding, a raw audio stream that is decoded and rendered and can be directly output to a speaker for playback, or data from a sensor or data used to drive a 5D cockpit experience.

[0101] Frame check field: used to check the entire frame data. The specific check algorithm can be selected from various mainstream check algorithms such as sum check and CRC check.

[0102] It should be noted that the above audio data transmission protocol may include but is not limited to: Figure 2 The parts shown may also include a protocol version number, etc. In addition, the order of various data in the audio data transmission protocol may also be adjusted.

[0103] The embodiment of the present application provides a specific format of a target audio transmission protocol, which carries and records different types of audio code streams and transmission modes by setting multiple data unit fields and data unit quantity fields. The format has the following advantages:

[0104] On the one hand, the format can support multiple types of audio streams and transmission methods, avoiding transmission function restrictions or incompatibility caused by fixed audio transmission protocol formats. For example, for processing nodes of different types or versions, the supported audio stream types and transmission methods may vary. If a fixed audio transmission protocol format is used, it may lead to transmission function restrictions or incompatibility. The format can carry different types of audio streams by setting multiple data unit fields, and distinguish different types of audio streams by setting type labels; at the same time, the format can support audio streams of different lengths and forms by setting data unit lengths and data segments, and record the number of different types of audio streams by setting the number of data units.

[0105] On the other hand, this format can improve the transmission efficiency and quality of audio code streams, and avoid communication delays or distortions caused by unreasonable or inappropriate audio transmission protocol formats. For example, for certain types or versions of processing nodes, the types of audio code streams and transmission methods they support may have advantages and disadvantages. If an unreasonable or inappropriate audio transmission protocol format is used, communication delays or distortions may occur. This format can ensure the integrity and continuity of the audio code stream by setting a reasonable data unit length and data segment, and ensure the balance and stability of the audio code stream by setting an appropriate number of data units.

[0106] In an optional implementation manner, the audio code stream carried in the above data unit field includes at least one of the following:

[0107] Encode and compress the audio code stream;

[0108] Multi-channel and object streams with metadata after local decoding;

[0109] After decoding, it is rendered and can be directly output to the original audio stream played by the speaker;

[0110] Sensor data in different forms.

[0111] An embodiment of the present application provides a specific method for supporting different types of audio code streams and transmission methods. This method carries different types of audio code streams by setting different types of data unit fields, and distinguishes different types of audio code streams by setting type labels.

[0112] By adopting the embodiment of the present application, the above method can support multiple types of audio streams and transmission methods, avoiding transmission function restrictions or incompatibility caused by fixed audio stream types and transmission methods. For example, for processing nodes of different types or versions, the supported audio stream types and transmission methods may be different. If fixed audio stream types and transmission methods are used, it may lead to transmission function restrictions or incompatibility. In the embodiment of the present application, different types of data unit fields are set to carry at least one of the following audio streams: encoded and compressed audio streams; multi-channel and object streams with metadata after local decoding; audio streams that are rendered after decoding and can be directly output to the speaker for playback; sensor data in different forms.

[0113] In addition, the above-mentioned embodiments of the present application can also improve the transmission efficiency and quality of the audio code stream, and avoid communication delays or distortions caused by unreasonable or inappropriate audio code stream types and transmission methods. For example, for certain types or versions of processing nodes, the audio code stream types and transmission methods they support may have advantages and disadvantages. If unreasonable or inappropriate audio code stream types and transmission methods are used, communication delays or distortions may occur. Using the embodiments of the present application, the integrity and continuity of the audio code stream can be guaranteed by setting a reasonable data unit length and data segment, and the recognizability and distinguishability of the audio code stream can be guaranteed by setting an appropriate type label.

[0114] As an optional embodiment, an optional protocol layer of the target audio transmission protocol is as follows: Figure 3 As shown, the physical layer may be a synchronous audio link such as I2S or A2B physical layer, or a general physical network link such as Ethernet. The link layer may be a link protocol supporting the physical layer, such as A2B, etc., which has a corresponding link protocol. The target audio transmission protocol in the embodiment of the present application is located at the application layer to support various audio applications.

[0115] In the embodiment of the present application, the flexible check field setting in the audio transmission protocol is conducive to realizing a low-delay error retransmission mechanism between nodes. Based on the target audio data protocol, the master node or the slave node can realize computing load balancing between different processor nodes. Based on the target audio data protocol, the master node or the slave node can receive the user's input, transmit it between nodes, and output it to different types of terminal nodes, thereby bringing users a multi-dimensional experience including audio.

[0116] In an optional implementation manner, the master node is further used to perform at least one of the following processing on the audio code stream: decoding, rendering, and post-processing to obtain a processed audio code stream.

[0117] An embodiment of the present application also provides a specific method for decoding an audio code stream. The method decodes the audio code stream by adopting a decoding algorithm for spatial audio processing to obtain spatial audio data and metadata. The metadata includes object metadata and multi-dimensional perception metadata, for example, five-dimensional 5D cockpit metadata.

[0118] Optionally, in one example, object metadata is for auditory experience, such as the XYZ coordinates of the sound, rendering angle, gain, etc.; 5D cockpit metadata is for other physical experiences, such as auxiliary auditory experience through seat vibration, flashing lights, aromatherapy spraying, etc.

[0119] For example, for some audio signal types that need to have a sense of space and three-dimensionality, such as surround sound, three-dimensional stereo, etc., this method can restore its original spatial information and effects through a spatial audio decoding algorithm and convert it into spatial audio data and metadata. By adopting the embodiment of the present application, the sense of space and three-dimensionality of the audio signal can be improved, avoiding the bland or monotonous sound quality caused by ordinary audio decoding algorithms.

[0120] For another example, different types or versions of vehicle-mounted nodes or power amplifier nodes may support different spatial audio decoding algorithms. If these differences are not taken into account, it may lead to sound quality differences or distortion. In the embodiment of the present application, a unified spatial audio decoding algorithm can be adopted to ensure the consistency and compatibility between various nodes, thereby improving the adaptability and compatibility of audio signals and avoiding sound quality differences or distortion caused by different types or versions of vehicle-mounted nodes or power amplifier nodes.

[0121] In an optional embodiment, the slave node includes at least one of the following:

[0122] The power amplifier slave node is used to perform at least one of the following processing on the above audio code stream: decoding, rendering, and post-processing to obtain a processed audio code stream;

[0123] The multi-dimensional perception drive slave node is used to perform drive control processing on the multi-dimensional perception metadata obtained by audio decoding processing to obtain multi-dimensional perception drive control data.

[0124] In an optional implementation, a multidimensional perception drive slave node among the above slave nodes is used to perform drive control processing on the multidimensional perception metadata obtained by audio decoding processing to obtain multidimensional perception drive control data, including at least one of the following:

[0125] The ambient light driving slave node is used to perform driving control processing on the ambient light data related to the above-mentioned spatial audio data to obtain the ambient light driving control data.

[0126] The vibration drive slave node is used to perform drive control processing on the vibration data related to the above-mentioned spatial audio data to obtain seat vibration drive control data.

[0127] The fragrance drive slave node is used to perform drive control processing on the fragrance data related to the above-mentioned spatial audio data to obtain fragrance drive control data.

[0128] The air-conditioning driving slave node is used to perform driving control processing on the air-conditioning data related to the above-mentioned spatial audio data to obtain air-conditioning driving control data.

[0129] The glasses drive slave node is used to perform drive control processing on the video data related to the above-mentioned spatial audio data to obtain audio-visual drive control data.

[0130] The sensor slave node is used to transmit the received sensor data to the above-mentioned power amplifier slave node, so that the above-mentioned power amplifier slave node performs multi-channel rendering processing on the above-mentioned spatial audio data based on the above-mentioned object metadata and the above-mentioned sensor data to obtain the rendered audio data.

[0131] The embodiment of the present application also provides a specific structure and function of a distributed audio processing system, which includes one or more different types of slave nodes, such as power amplifier slave nodes, multi-dimensional perception drive slave nodes, for example, the multi-dimensional perception drive slave nodes may include but are not limited to: atmosphere light drive slave nodes, vibration drive slave nodes, fragrance drive slave nodes, air conditioning drive slave nodes, glasses drive slave nodes, sensor slave nodes, etc. These slave nodes are responsible for encoding, rendering and other processing of spatial audio data according to different types of metadata or sensor data, or driving and controlling multi-dimensional perception metadata related to spatial audio data to obtain multi-dimensional perception drive control data. For example, the above-mentioned multi-dimensional perception metadata includes but is not limited to: video data (including but not limited to brightness, frame rate, resolution, etc.), vibration data (including but not limited to position, intensity, etc.), atmosphere light data (including but not limited to color, brightness, etc.), fragrance data (including but not limited to odor type / name, air flow rate, humidity, temperature, etc.), air conditioning data (including but not limited to temperature, humidity, wind direction and wind speed, etc.).

[0132] For example, for some audio code stream types that require more computing resources or special functions, such as spatial audio, three-dimensional stereo, etc., the embodiments of the present application realize rendering, driving or other processing through collaboration between different types of slave nodes, and transmit the processed audio code stream to the speaker for playback; for some audio code stream types that require fewer computing resources or ordinary functions, such as ordinary stereo, mono, etc., the system can realize functions such as amplification and output through a single type of slave node, and transmit the processed audio code stream to the speaker for playback. In this way, the functions and performance of the distributed audio processing system can be improved, avoiding functional limitations or poor performance caused by a single slave node.

[0133] For another example, different types or versions of slave nodes may support different audio code stream types and processing functions. If a single type of slave node is used, it may cause functional mismatch or incompatibility. In the embodiment of the present application, the adaptation and compatibility of slave nodes of different types or versions are achieved through negotiation and coordination between different types of slave nodes, thereby improving the adaptability and compatibility of the distributed audio processing system and avoiding functional mismatch or incompatibility caused by different types or versions of slave nodes.

[0134] Since the current audio playback framework of the car operating system is for traditional stereo and surround sound, and does not take into account the immersive and interactive rendering and playback characteristics of multi-dimensional spatial audio, both the interface and the playback link need to be optimized. Therefore, the embodiment of the present application proposes a first audio subsystem with low latency and high reliability that natively supports a multi-dimensional spatial audio interface.

[0135] In an optional implementation manner, the master node is provided with a first audio subsystem supporting a spatial audio interface, and the first audio subsystem includes:

[0136] The first input layer is used to receive the above audio code streams from different applications and convert the above audio code streams into audio data packets in a unified format;

[0137] A mixing layer, used for mixing and sorting the audio data packets transmitted by the first input layer to obtain mixed audio data packets;

[0138] A first output layer, used for distributing the mixed audio data packets output by the mixing layer to the slave nodes or the terminal nodes;

[0139] The first configuration layer is used to configure the parameters and functions of the first input layer, the mixing layer and the first output layer according to different playback environments and user requirements.

[0140] In an embodiment of the present application, in a master node, a specific structure and function of a first audio subsystem supporting a spatial audio interface is also provided, wherein the first subsystem includes a first input layer, a mixing layer, a first output layer, and a first configuration layer. The first input layer can be used to receive audio code streams from different applications and convert them into audio data packets in a unified format; the mixing layer is responsible for mixing and sorting the audio data packets transmitted by the input layer to obtain mixed audio data packets; the first output layer can be used to distribute the mixed audio data packets output by the mixing layer to slave nodes or terminal nodes; the first configuration layer can be used to configure the parameters and functions of the first input layer, the mixing layer, and the first output layer according to different playback environments and user needs.

[0141] In one example, different types or versions of applications may support different audio code stream types and processing methods. If fixed audio code stream types and processing methods are used, processing functions may be limited or incompatible. The first audio subsystem can receive multiple types of audio code streams through the first input layer and convert them into audio data packets in a unified format. At the same time, the first audio subsystem can configure different types of processing methods corresponding to different types of audio code streams through the first configuration layer, such as decoding, rendering, post-processing, etc.

[0142] As a result, the master node can support multiple types of audio code streams and processing methods, avoiding processing function limitations or incompatibility caused by fixed audio code stream types and processing methods.

[0143] In another example, for example, for certain types or versions of applications, the supported audio code stream types and processing methods may have advantages and disadvantages. If unreasonable or inappropriate audio code stream types and processing methods are used, processing delays or distortions may occur. The first audio subsystem can be used to mix and sort the audio data packets transmitted by the first input layer through the mixing layer to obtain mixed audio data packets, and the mixed audio data packets output by the mixing layer are distributed to slave nodes or terminal nodes through the first output layer.

[0144] As a result, the master node can improve the processing efficiency and quality of the audio code stream and avoid processing delays or distortions caused by unreasonable or inappropriate audio code stream types and processing methods.

[0145] In addition, if Figure 4 As shown, the master node or the slave node may also include, but is not limited to, one or more processing modules:

[0146] The thread optimization module is used to achieve low-latency and high-reliability data transmission between the master node and the slave node. The thread optimization module optimizes data transmission by dynamically adjusting the number and priority of threads.

[0147] The embodiment of the present application realizes low-latency and high-reliability data transmission between the master node and the slave node by setting a thread optimization module, optimizes data transmission by dynamically adjusting the number and priority of threads, realizes fast transmission and synchronization of audio data, and improves the transmission efficiency and reliability of audio data.

[0148] The cache optimization module is used to realize efficient storage and reading of data between the master node and the slave node. The cache optimization module realizes the optimization of data storage and reading by dynamically adjusting the cache size and position.

[0149] In the embodiment of the present application, a cache optimization module is set up to optimize data storage and reading by dynamically adjusting the cache size and position, so that efficient storage and reading of audio data can be achieved, and the memory usage and reading time of audio data are reduced.

[0150] The cross-layer optimization module is used to realize fast conversion and processing of data between the master node and the slave node. The cross-layer optimization module realizes optimization of data conversion and processing by dynamically adjusting the format and interface of the data.

[0151] The embodiment of the present application optimizes data conversion and processing by setting a cross-layer optimization module between the master node and the slave node by dynamically adjusting the format and interface of the data, thereby achieving rapid conversion and processing of audio data.

[0152] The spatial audio production module is used to implement spatial processing of audio data on the master node or the slave node, and the spatial audio production module implements the spatialization of audio data by adding metadata to the audio data. For example, in the embodiment of the present application, the spatial audio can be but is not limited to three-dimensional spatial audio, and can also be a one-dimensional stereo or two-dimensional surround sound audio signal.

[0153] In the embodiment of the present application, by setting up a spatial audio production module, native support for spatial audio can be achieved, the immersion and interactivity of audio data can be improved, spatial processing of audio data can be achieved, and the spatial sense and positioning sense of audio data can be improved.

[0154] The intelligent tuning module is used to implement post-processing equalization of audio data on a master node or a slave node. The intelligent tuning module implements post-processing equalization of audio data by analyzing and adjusting the audio data.

[0155] In the embodiment of the present application, by setting up an intelligent tuning module, post-processing equalization of audio data can be achieved, thereby improving the balance and adaptability of audio data; different forms of processing of audio data can be achieved, thereby improving the diversity and compatibility of audio data.

[0156] In the embodiments of the present application, the characteristics of low latency and high reliability are achieved through technical means such as thread optimization, cache optimization, cross-layer optimization (cross-layer call optimization), and spatial audio production capabilities and intelligent tuning are provided.

[0157] Optionally, the interfaces between different layers use a unified packet format. There are many types of packet formats, but in principle they must contain three types of content: label, length, and information. A complete package can contain one or more of these three sets, and the information itself can also be decomposed into the next level of label, length, and information.

[0158] For example, each interface is a string, the producer is responsible for framing, and the consumer is responsible for deframing. Each playback environment has a configuration string: Type, Length, payload. This basic format is conducive to forward and backward compatibility.

[0159] In addition, in the embodiment of the present application, the functions of each layer can be processed by the master node or the slave node according to the different external devices associated with the slave node, and then adapt to different automobile hardware through the above-mentioned audio data transmission protocol. Optionally, the above-mentioned external devices can include but are not limited to: audio input devices, such as microphones; audio output devices, such as speakers; auxiliary input devices, such as mice, keyboards, and joysticks; auxiliary output devices, such as seats, air conditioners, and lights.

[0160] In an optional implementation manner, a corresponding second audio subsystem is provided on each of the slave nodes, and the second audio subsystem includes:

[0161] The second input layer is used to receive the audio code stream from the master node or other slave nodes, and convert the audio code stream into an audio data packet in a unified format;

[0162] A processing layer, used for performing the slave node processing on the audio data packet transmitted by the second input layer to obtain a processed audio data packet;

[0163] The second output layer is used to distribute the processed audio data packets output by the processing layer to other slave nodes or the terminal nodes;

[0164] The second configuration layer is used to configure the parameters and functions of the second input layer, the processing layer and the second output layer according to different playback environments and user requirements.

[0165] An embodiment of the present application also provides a specific structure and function of a second audio subsystem that supports multiple types of slave node processing methods. The second audio subsystem includes: a second input layer, a processing layer, a second output layer and a second configuration layer. The second input layer is used to receive audio code streams from a master node or other slave nodes and convert them into audio data packets in a unified format; the processing layer is used to perform slave node processing on the audio data packets transmitted by the second input layer to obtain processed audio data packets; the second output layer is used to distribute the processed audio data packets output by the processing layer to other slave nodes or terminal nodes; the second configuration layer is responsible for configuring the parameters and functions of the second input layer, the processing layer and the second output layer according to different playback environments and user needs.

[0166] For example, for different types or versions of slave nodes, the slave node processing methods they support may be different. If a fixed slave node processing method is used, it may result in limited or incompatible processing functions. The second audio subsystem in the embodiment of the present application can configure different types of slave node processing methods corresponding to different types of slave nodes through a configuration layer, such as rendering, driving, amplification, output, etc. On the one hand, the subsystem can support multiple types of slave node processing methods to avoid limited or incompatible processing functions caused by fixed slave node processing methods.

[0167] On the other hand, the subsystem can improve the processing efficiency and quality of the slave nodes, and avoid processing delays or distortions caused by unreasonable or inappropriate slave node processing methods. For example, for certain types or versions of slave nodes, the slave node processing methods they support may have advantages and disadvantages. If unreasonable or inappropriate slave node processing methods are used, processing delays or distortions may occur. The subsystem can perform slave node processing on the audio data packets transmitted by the second input layer through the processing layer to obtain processed audio data packets, and distribute the processed audio data packets output by the processing layer to other slave nodes or terminal nodes through the second output layer.

[0168] In one example, Figure 5 A schematic diagram of an optional distributed 5D spatial audio cockpit system is shown, as shown in Figure 5 As shown, in an optional 5D spatial audio cockpit solution, after the spatial audio multimedia content is decoded by the spatial audio of the host node, spatial audio data and object metadata will be generated, as well as metadata supporting the 5D cockpit experience. All of these data can be transmitted on the transmission bus through the aforementioned audio transmission protocol and distributed to various different slave nodes for corresponding targeted processing.

[0169] For example, the power amplifier slave node can extract multi-channel audio data and object metadata from the transmission bus, and after spatial audio rendering processing, transmit the rendered data to the vehicle speakers; at the same time, it can also transmit the audio bitstream and object metadata, as well as videos obtained through other channels such as wireless networks to the XR glasses; it can also send various 5D metadata such as force, heat, light, electricity, etc. to related slave nodes to enhance the user's 5D cockpit experience.

[0170] In addition, in the embodiment of the present application, sensor data transmitted from sensors such as speed and direction can also be transmitted to rendering nodes through relevant slave nodes to achieve a more interactive spatial audio experience. For example, a node that converts sensor data into spatial position information, a node that converts sensor data into spatial acoustic model data, and so on.

[0171] Therefore, by adopting the distributed audio processing system provided in the embodiment of the present application, it is relatively easy to realize personalized upgrades and iterations of products, thereby extending the product life cycle and improving the user's product experience.

[0172] The present application also provides an embodiment of a distributed audio processing method. Figure 6 is a flow chart of a distributed audio processing method provided by an embodiment of the present application, such as Figure 6 As shown, the method includes:

[0173] S101, obtaining an audio processing task received by a distributed audio processing system, wherein data is transmitted between at least two processing nodes in the distributed audio processing system based on a target audio transmission protocol.

[0174] S102: Identify the processing type of the audio code stream in the audio processing task according to the type field in the target audio transmission protocol.

[0175] S103, transmitting the audio code stream to a target processing node corresponding to the processing type among the at least two processing nodes, so that the target processing node is used to process the audio code stream, and the processed audio code stream is transmitted to at least one speaker for playback.

[0176] Optionally, in an embodiment of the present application, the above-mentioned distributed audio processing method can be run in a distributed audio processing system, which is a vehicle-mounted audio processing system. In this system, the transmission bus is also used to receive audio processing tasks, and identify and transmit different types of audio code streams according to the type field in the target audio transmission protocol; and at least two processing nodes include a master node and at least one slave node, and the master node and at least one slave node are connected by a transmission bus, and the transmission bus is used to transmit the audio code stream to be processed and the processed audio code stream.

[0177] Optionally, in an embodiment of the present application, an audio transmission protocol is run on the transmission bus between the master node and the slave node to specify the format and transmission method of the audio code stream and other related data, thereby achieving effective distribution and reception of audio data.

[0178] In an embodiment of the present application, the master node is a device for receiving, processing and transmitting audio signals, for example, such as spatial audio decoding, sound bed or object multi-channel rendering, post-processing equalization, etc.; the slave node is a device for special audio processing and outputting audio signals, for example, it can process spatial audio decoding, sound bed or object multi-channel rendering, post-processing equalization, multi-dimensional perception driving, etc.

[0179] In one example, the master node and the slave node allocate audio processing tasks based on the audio transmission protocol, according to the capabilities between the nodes or the needs of the application. For example, the audio processing business can be allocated according to factors such as computing power, memory space, and hardware power consumption. If a digital signal processing DSP chip has sufficient computing power resources and strong real-time performance, it can perform rendering and sound field calibration, and the workload of the subsequent power amplifier node will be very small; otherwise, the sound field calibration work may be handed over to the power amplifier node, and the DSP information is only rendered. Finally, the processed audio stream is output to one or more speakers through the terminal node for playback.

[0180] In the embodiment of the present application, the form of the audio code stream is identified according to the type field, and different audio processing processes are adopted to realize the processing of different forms of audio data; according to different types of audio code streams and transmission methods, a suitable processing node is dynamically selected to process the audio code stream, and the processed audio code stream is transmitted to the speaker for playback. It can also drive seats, ambient lights, air conditioners and other facilities, thereby realizing the output and driving of audio data.

[0181] In one example, after receiving the audio code stream, the master node can transmit the processed or unprocessed audio code stream of the master node to the slave node based on the audio transmission protocol, regardless of whether to process it, so that the slave node processes the received audio code stream to obtain a processed audio code stream, or directly transmits the audio code stream processed by the master node as the above-mentioned processed audio code stream to at least one speaker for playback, and drives seats, ambient lights, air conditioners and other facilities, thereby realizing the output and driving of audio data.

[0182] In one example, for example, for some audio code stream types that require more computing resources or special functions, such as spatial audio, 3D sound, etc., the method can transfer them to a target processing node with corresponding computing power or function for processing, thereby improving processing speed and quality; for some audio code stream types that require less computing resources or ordinary functions, such as ordinary stereo, mono, etc., the method can transfer them to a target processing node with lower computing power or function for processing, thereby saving computing resources and power. This can improve the resource utilization and performance of the distributed audio processing system and avoid resource waste or shortage caused by fixed processing node selection.

[0183] In another example, for example, different types or versions of processing nodes may support different audio code stream types and transmission methods. If these differences are not taken into account, it may lead to sound quality differences or distortion. This method can determine the processing type of the audio code stream by identifying the type field in the target audio transmission protocol, and select a suitable target processing node to process the audio code stream based on this. In turn, the adaptability and compatibility of the distributed audio processing system can be improved, avoiding sound quality differences or distortion caused by different types or versions of processing nodes.

[0184] Optionally, the above-mentioned audio transmission protocol is used to specify the format and transmission method of audio code streams and other related data, realizing the effective distribution and reception of audio data to meet the different deployment requirements of audio processing systems and achieve load balancing between different processor nodes.

[0185] In an optional embodiment, after receiving the audio code stream, the master node generally performs some processing, for example: obtaining the audio code stream and / or 5D data from the system stream by demultiplexing; further parsing the audio code stream, such as parsing out the sound bed, object or scene audio signal; or further partially or completely decoding the parsed audio signal, such as partially or completely decoding the compressed sound bed, object or scene signal, such as decoding it into a PCM signal, a PCM+Metadata signal, or a PCM array signal; or further rendering part or all of the decoded signal to obtain the signal of the target speaker; or further multiplexing the input information of other sensors such as cameras and radars with the audio data of the above different stages, and distributing the audio code stream processed by the master node to other slave nodes in the system, generally dedicated audio processing nodes, for further processing according to a predetermined audio transmission protocol.

[0186] In another embodiment, the embodiment of the present application also supports distributed interaction between nodes: the master node or the slave node can receive user input or data input from various related sensors, and can transmit relevant data between nodes, and these data and audio data are used for fusion rendering; the multi-processor node system that communicates through the transmission protocol is beneficial for load balancing of audio processing computing power, fusion calculation of multiple sensors, collaborative work of multiple drive units, local optimization and upgrading of the system, etc.

[0187] Optionally, the above-mentioned audio data transmission protocol can transmit sensor data in different forms, such as data collected by the Automatic Data Acquisition System (ADAS), speed sensor, etc., which can be transmitted and aggregated into a data stream through the transmission bus and received and processed by each processing node on the bus.

[0188] An optional implementation method, Figure 7 is a flow chart of an optional distributed audio processing method provided in an embodiment of the present application, such as Figure 7 As shown, transmitting the audio code stream to a target processing node corresponding to the processing type among the at least two processing nodes includes:

[0189] S201, determining an audio processing requirement corresponding to the above processing type;

[0190] S202, determining the target processing node for processing the audio code stream according to the audio processing requirement corresponding to the processing type;

[0191] S203, transmitting the audio code stream to the target processing node.

[0192] The embodiment of the present application also provides a specific method for selecting a target processing node, which determines different types of audio processing requirements corresponding to different types of audio code streams, and determines the target processing node for processing the type of audio code stream based on this.

[0193] In an optional embodiment, for example, for some audio bitstream types that require more computing resources or special functions, such as spatial audio, 3D sound, etc., this method can determine the target processing node with corresponding computing power or function according to its corresponding audio processing requirements, such as decoding, rendering, post-processing, etc., so as to ensure the excellent and stable processing effect; for some audio bitstream types that require less computing resources or ordinary functions, such as ordinary stereo, mono, etc., this method can determine the target processing node with lower computing power or function according to its corresponding audio processing requirements, such as amplification, output, etc., so as to ensure the simplicity and speed of the processing effect. Furthermore, the target processing node can be accurately selected to avoid poor or unstable processing effect caused by selection error of the target processing node.

[0194] In another optional embodiment, for example, for abnormal situations occurring at certain time points, such as failure, freezing, restarting, etc. of a processing node, this method can timely adjust the selection of the target processing node according to the actual situation, thereby ensuring the normal processing of the audio code stream. Furthermore, the target processing node can be flexibly selected to avoid waste or shortage of resources caused by fixed target processing node selection.

[0195] In an optional implementation manner, the protocol data format of the above-mentioned target audio transmission protocol at least includes:

[0196] Multiple data unit fields are used to carry audio code streams of different processing types. Each data unit field contains: type label, unit length, and data segment;

[0197] The data unit quantity field is used to record the number of processing types of the audio code stream in the audio transmission protocol frame.

[0198] Optionally, the target audio transmission protocol may be a panoramic sound WANOS protocol, in which one or more tags may be defined to indicate the processing type of the audio data; as an optional embodiment, the processing type may also be divided into multiple levels, such as "music class" --> "rock class" --> "pure WANOS object class", etc.

[0199] Optionally, the audio data format can be classified according to different angles, such as compression class / uncompression class, sound bed class / object class, etc.; the processing methods corresponding to different classification methods are also different. For example, the compression class needs to be decoded first, but the uncompression class does not need to be decoded; the sound bed class needs to perform the "channel signal->channel signal" rendering method, and the object class needs to perform the "channel signal+object metadata->channel signal" rendering method, etc.; it should be noted that in the embodiments of the present application, multiple classification methods can be combined to obtain more detailed categories.

[0200] In addition, as an optional embodiment, Figure 2 As shown, the above target audio transmission protocol may also include:

[0201] Frame header synchronization field: used by the receiving node to detect the beginning of a transmission protocol frame.

[0202] The frame length field is used to indicate the length of a frame.

[0203] Optional frame check 1: can be used to improve the reliability of frame header detection. The specific check algorithm can be selected from various mainstream check algorithms such as sum check and CRC check.

[0204] Number of data units: used to indicate how many different types of audio data are in this frame

[0205] Data unit: used to carry different types of audio data, generally composed of type label, unit length, data segment, data unit check, etc.

[0206] The data segment can carry different types of data, such as encoded and compressed audio streams, multi-channel and object streams with metadata after local decoding, raw audio streams that are decoded and rendered and can be directly output to speakers for playback, data from sensors, or data used to drive a 5D cockpit experience.

[0207] Frame check n: used to check the entire frame data. The specific check algorithm can be selected from various mainstream check algorithms such as sum check and CRC check.

[0208] It should be noted that the above audio data transmission protocol may include but is not limited to: Figure 2 The parts shown may also include a protocol version number, etc. In addition, the order of various data in the audio data transmission protocol may also be adjusted.

[0209] The embodiment of the present application provides a specific format of a target audio transmission protocol, which carries and records different types of audio code streams and transmission modes by setting multiple data unit fields and data unit quantity fields. The format has the following advantages:

[0210] On the one hand, the format can support multiple types of audio streams and transmission methods, avoiding transmission function restrictions or incompatibility caused by fixed audio transmission protocol formats. For example, for processing nodes of different types or versions, the supported audio stream types and transmission methods may vary. If a fixed audio transmission protocol format is used, it may lead to transmission function restrictions or incompatibility. The format can carry different types of audio streams by setting multiple data unit fields, and distinguish different types of audio streams by setting type labels; at the same time, the format can support audio streams of different lengths and forms by setting data unit lengths and data segments, and record the number of different types of audio streams by setting the number of data units.

[0211] On the other hand, this format can improve the transmission efficiency and quality of audio code streams, and avoid communication delays or distortions caused by unreasonable or inappropriate audio transmission protocol formats. For example, for certain types or versions of processing nodes, the types of audio code streams and transmission methods they support may have advantages and disadvantages. If an unreasonable or inappropriate audio transmission protocol format is used, communication delays or distortions may occur. This format can ensure the integrity and continuity of the audio code stream by setting a reasonable data unit length and data segment, and ensure the balance and stability of the audio code stream by setting an appropriate number of data units.

[0212] In an optional implementation manner, the audio code stream carried in the above data unit field includes at least one of the following:

[0213] Encode and compress the audio code stream;

[0214] Multi-channel and object streams with metadata after local decoding;

[0215] After decoding, it is rendered and can be directly output to the original audio stream played by the speaker;

[0216] Sensor data in different forms.

[0217] An embodiment of the present application provides a specific method for supporting different types of audio code streams and transmission methods. This method carries different types of audio code streams by setting different types of data unit fields, and distinguishes different types of audio code streams by setting type labels.

[0218] By adopting the embodiment of the present application, the above method can support multiple types of audio streams and transmission methods, avoiding transmission function restrictions or incompatibility caused by fixed audio stream types and transmission methods. For example, for processing nodes of different types or versions, the supported audio stream types and transmission methods may be different. If fixed audio stream types and transmission methods are used, it may lead to transmission function restrictions or incompatibility. In the embodiment of the present application, different types of data unit fields are set to carry at least one of the following audio streams: encoded and compressed audio streams; multi-channel and object streams with metadata after local decoding; audio streams that are rendered after decoding and can be directly output to the speaker for playback; sensor data in different forms.

[0219] In addition, the above-mentioned embodiments of the present application can also improve the transmission efficiency and quality of the audio code stream, and avoid communication delays or distortions caused by unreasonable or inappropriate audio code stream types and transmission methods. For example, for certain types or versions of processing nodes, the audio code stream types and transmission methods they support may have advantages and disadvantages. If unreasonable or inappropriate audio code stream types and transmission methods are used, communication delays or distortions may occur. Using the embodiments of the present application, the integrity and continuity of the audio code stream can be guaranteed by setting a reasonable data unit length and data segment, and the recognizability and distinguishability of the audio code stream can be guaranteed by setting an appropriate type label.

[0220] As an optional embodiment, the entire protocol layer of audio transmission is as follows: Figure 3 As shown, the physical layer may be a synchronous audio link such as I2S or A2B physical layer, or a general physical network link such as Ethernet. The link layer may be a link protocol supporting the physical layer, such as A2B, etc., which has a corresponding link protocol. The target audio transmission protocol in the embodiment of the present application is located at the application layer to support various audio applications.

[0221] An optional implementation manner, using the target processing node to process the audio code stream to obtain a processed audio code stream, includes:

[0222] The master node is used to perform at least one of the following processing on the audio code stream: decoding, rendering, and post-processing to obtain a processed audio code stream.

[0223] An embodiment of the present application also provides a specific method for decoding an audio code stream. The method decodes the audio code stream by adopting a decoding algorithm for spatial audio processing to obtain spatial audio data and metadata. The metadata includes object metadata and multi-dimensional perception metadata, for example, five-dimensional 5D cockpit metadata.

[0224] Optionally, in one example, object metadata is for auditory experience, such as the XYZ coordinates of the sound, rendering angle, gain, etc.; 5D cockpit metadata is for other physical experiences, such as auxiliary auditory experience through seat vibration, flashing lights, aromatherapy spraying, etc.

[0225] For example, for some audio signal types that need to have a sense of space and stereo, such as surround sound, 3D sound, etc., this method can restore its original spatial information and effects through a spatial audio decoding algorithm and convert it into spatial audio data and metadata. By adopting the embodiment of the present application, the sense of space and stereo of the audio signal can be improved, avoiding the bland or monotonous sound quality caused by ordinary audio decoding algorithms.

[0226] For another example, different types or versions of vehicle-mounted nodes or power amplifier nodes may support different spatial audio decoding algorithms. If these differences are not taken into account, it may lead to sound quality differences or distortion. In the embodiment of the present application, a unified spatial audio decoding algorithm can be adopted to ensure the consistency and compatibility between various nodes, thereby improving the adaptability and compatibility of audio signals and avoiding sound quality differences or distortion caused by different types or versions of vehicle-mounted nodes or power amplifier nodes.

[0227] In an optional embodiment, the target processing node is used to process the audio code stream to obtain a processed audio code stream, including:

[0228] S301, using a power amplifier slave node among the slave nodes to perform at least one of the following processing on the audio code stream: decoding, rendering, and post-processing to obtain a processed audio code stream;

[0229] S302, using the multi-dimensional perception driving slave node among the above slave nodes, to perform driving control processing on the multi-dimensional perception metadata obtained by audio decoding processing, and obtain multi-dimensional perception driving control data.

[0230] In an optional implementation, a multidimensional perception drive slave node among the above slave nodes is used to perform drive control processing on the multidimensional perception metadata obtained by audio decoding processing to obtain multidimensional perception drive control data, including at least one of the following:

[0231] The ambient light driving slave node is used to perform driving control processing on the ambient light data related to the above-mentioned spatial audio data to obtain the ambient light driving control data.

[0232] The vibration drive slave node is used to perform drive control processing on the vibration data related to the above-mentioned spatial audio data to obtain seat vibration drive control data.

[0233] The fragrance drive slave node is used to perform drive control processing on the fragrance data related to the above-mentioned spatial audio data to obtain fragrance drive control data.

[0234] The air-conditioning driving slave node is used to perform driving control processing on the air-conditioning data related to the above-mentioned spatial audio data to obtain air-conditioning driving control data.

[0235] The glasses drive slave node is used to perform drive control processing on the video data related to the above-mentioned spatial audio data to obtain audio-visual drive control data.

[0236] The sensor slave node is used to transmit the received sensor data to the above-mentioned power amplifier slave node, so that the above-mentioned power amplifier slave node performs multi-channel rendering processing on the above-mentioned spatial audio data based on the above-mentioned object metadata and the above-mentioned sensor data to obtain the rendered audio data.

[0237] The embodiment of the present application also provides a specific structure and function of a distributed audio processing system, which includes one or more different types of slave nodes, such as power amplifier slave nodes, multi-dimensional perception drive slave nodes, for example, the multi-dimensional perception drive slave nodes may include but are not limited to: atmosphere light drive slave nodes, vibration drive slave nodes, fragrance drive slave nodes, air conditioning drive slave nodes, glasses drive slave nodes, sensor slave nodes, etc. These slave nodes are responsible for encoding, rendering and other processing of spatial audio data according to different types of metadata or sensor data, or driving and controlling multi-dimensional perception metadata related to spatial audio data to obtain multi-dimensional perception drive control data. For example, the above-mentioned multi-dimensional perception metadata includes but is not limited to: video data, vibration data, atmosphere light data, fragrance data, air conditioning data (including temperature, humidity, wind direction and wind speed, etc.).

[0238] For example, for some audio code stream types that require more computing resources or special functions, such as spatial audio, three-dimensional stereo, etc., the embodiments of the present application realize rendering, driving or other processing through collaboration between different types of slave nodes, and transmit the processed audio code stream to the speaker for playback; for some audio code stream types that require fewer computing resources or ordinary functions, such as ordinary stereo, mono, etc., the system can realize functions such as amplification and output through a single type of slave node, and transmit the processed audio code stream to the speaker for playback. In this way, the functions and performance of the distributed audio processing system can be improved, avoiding functional limitations or poor performance caused by a single slave node.

[0239] For another example, different types or versions of slave nodes may support different audio code stream types and processing functions. If a single type of slave node is used, it may cause functional mismatch or incompatibility. In the embodiment of the present application, the adaptation and compatibility of slave nodes of different types or versions are achieved through negotiation and coordination between different types of slave nodes, thereby improving the adaptability and compatibility of the distributed audio processing system and avoiding functional mismatch or incompatibility caused by different types or versions of slave nodes.

[0240] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0241] According to one or more embodiments of the present application, a distributed audio processing device is provided. Figure 8 A structural block diagram of a distributed audio processing device provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the above device comprises:

[0242] An acquisition module 601 is used to acquire an audio processing task received by a distributed audio processing system, wherein data is transmitted between at least two processing nodes in the distributed audio processing system based on a target audio transmission protocol;

[0243] An identification module 602, configured to identify the processing type of the audio code stream in the audio processing task according to the type field in the target audio transmission protocol;

[0244] The processing module 603 is used to transmit the above-mentioned audio code stream to the target processing node corresponding to the above-mentioned processing type among the above-mentioned at least two processing nodes, so that the above-mentioned audio code stream is processed by the above-mentioned target processing node, and the obtained processed audio code stream is transmitted to at least one speaker for playback.

[0245] According to one or more embodiments of the present application, the processing module includes:

[0246] A first determining unit, used to determine an audio processing requirement corresponding to the above processing type;

[0247] A second determining unit, configured to determine the target processing node for processing the audio code stream according to the audio processing requirement corresponding to the processing type;

[0248] The transmission unit is used to transmit the above audio code stream to the above target processing node.

[0249] In an optional implementation manner, the protocol data format of the above-mentioned target audio transmission protocol at least includes:

[0250] Multiple data unit fields are used to carry audio code streams of different processing types. Each data unit field contains: type label, unit length, and data segment;

[0251] The data unit quantity field is used to record the number of processing types of the audio code stream in the audio transmission protocol frame.

[0252] In an optional implementation manner, the audio code stream carried in the above data unit field includes at least one of the following:

[0253] Encode and compress the audio code stream;

[0254] Multi-channel and object streams with metadata after local decoding;

[0255] After decoding, it is rendered and can be directly output to the original audio stream played by the speaker;

[0256] Sensor data in different forms.

[0257] In an optional implementation manner, the at least two processing nodes include: a master node and at least one slave node, and the processing module includes:

[0258] A first processing unit, configured to use the master node to perform at least one of the following processing on the audio code stream: decoding, rendering, and post-processing to obtain a processed audio code stream;

[0259] A second processing unit is used to use the power amplifier slave node in the slave node to perform at least one of the following processing on the audio code stream: decoding, rendering, and post-processing to obtain a processed audio code stream;

[0260] The driving control unit is used to use the multi-dimensional perception driving slave node in the above-mentioned slave nodes to perform driving control processing on the multi-dimensional perception metadata obtained by audio decoding processing to obtain multi-dimensional perception driving control data.

[0261] In an optional implementation manner, the drive control unit is further configured to perform at least one of the following drive controls:

[0262] The ambient light driving slave node is used to perform driving control processing on the ambient light data related to the above-mentioned spatial audio data to obtain the ambient light driving control data.

[0263] The vibration drive slave node is used to perform drive control processing on the vibration data related to the above-mentioned spatial audio data to obtain seat vibration drive control data.

[0264] The fragrance drive slave node is used to perform drive control processing on the fragrance data related to the above-mentioned spatial audio data to obtain fragrance drive control data.

[0265] The air-conditioning driving slave node is used to perform driving control processing on the air-conditioning data related to the above-mentioned spatial audio data to obtain air-conditioning driving control data.

[0266] The glasses drive slave node is used to perform drive control processing on the video data related to the above-mentioned spatial audio data to obtain audio-visual drive control data.

[0267] In an exemplary embodiment, the present application also provides an electronic device, including: a processor, and a memory connected to the processor;

[0268] The memory stores computer-executable instructions;

[0269] The processor executes the computer-executable instructions stored in the memory to implement any of the above methods.

[0270] In an exemplary embodiment, the embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement any of the above methods.

[0271] In an exemplary embodiment, the present application also provides a computer program product, including a computer program, which implements any of the above methods when executed by a processor.

[0272] In order to implement the above embodiment, the present application embodiment also provides an electronic device. Fig. 9 , which shows a schematic diagram of the structure of an electronic device 700 suitable for implementing the embodiment of the present application, and the electronic device 700 may be a terminal device or a server. The terminal device may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, message transceiver devices, game consoles, medical equipment, fitness equipment, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig. 9The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0273] like Fig. 9 As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 to a random access memory (RAM) 703. Various programs and data required for the operation of the electronic device 700 are also stored in the RAM 703. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0274] Typically, the following devices may be connected to the I / O interface 705: input devices 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 708 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 709. The communication devices 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although Fig. 9 The electronic device 700 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0275] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present application are executed.

[0276] It should be noted that the computer-readable medium mentioned above in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0277] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0278] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0279] The computer program code for performing the operation of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0280] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0281] The units involved in the embodiments described in the present application may be implemented by software or hardware. The name of the unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0282] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0283] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0284] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0285] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A distributed audio processing method, characterized in that: The method comprises: Acquire an audio processing task received by a distributed audio processing system, wherein data is transmitted between at least two processing nodes in the distributed audio processing system based on a target audio transmission protocol; Identify the processing type of the audio code stream in the audio processing task according to the type field in the target audio transmission protocol; The audio code stream is transmitted to a target processing node corresponding to the processing type among the at least two processing nodes, so that the audio code stream is processed by the target processing node, and the processed audio code stream is transmitted to at least one speaker for playback.

2. The method according to claim 1, characterized in that Transmitting the audio code stream to a target processing node corresponding to the processing type among the at least two processing nodes includes: determining a processing requirement corresponding to the processing type; Determining the target processing node for processing the audio code stream according to the processing requirements corresponding to the processing type; The audio code stream is transmitted to the target processing node.

3. The method according to claim 1, characterized in that The protocol data format of the target audio transmission protocol includes at least: Multiple data unit fields are used to carry audio code streams of different processing types. Each data unit field contains: type label, unit length, and data segment; The data unit quantity field is used to record the number of processing types of the audio code stream in the audio transmission protocol frame.

4. The method according to claim 3, characterized in that The audio code stream carried in the data unit field includes at least one of the following: Encode and compress the audio code stream; Multi-channel and object streams with metadata after local decoding; After decoding, it is rendered and can be directly output to the original audio stream played by the speaker; Sensor data in different forms.

5. The method according to claim 1, characterized in that The at least two processing nodes include: a master node and at least one slave node, and the target processing node is used to process the audio code stream to obtain a processed audio code stream, including: Using the master node to perform at least one of the following processing on the audio code stream: decoding, rendering, and post-processing to obtain a processed audio code stream; Using a power amplifier slave node among the slave nodes, the audio code stream is processed by at least one of the following: decoding, rendering, and post-processing to obtain a processed audio code stream; A multidimensional perception driving slave node among the slave nodes is used to perform driving control processing on the multidimensional perception metadata obtained by audio decoding processing to obtain multidimensional perception driving control data.

6. The method according to claim 5, characterized in that Using a multidimensional perception drive slave node among the slave nodes to perform drive control processing on the multidimensional perception metadata obtained by audio decoding processing to obtain multidimensional perception drive control data, including at least one of the following: Using an ambient light drive slave node to perform drive control processing on the ambient light data related to the spatial audio data to obtain ambient light drive control data; Using a vibration drive slave node to perform drive control processing on the vibration data related to the spatial audio data to obtain seat vibration drive control data; Using a fragrance drive slave node to perform drive control processing on the fragrance data related to the spatial audio data to obtain fragrance drive control data; Using an air-conditioning driving slave node to perform driving control processing on the air-conditioning data related to the spatial audio data to obtain air-conditioning driving control data; The glasses driving slave node is used to perform driving control processing on the video data related to the spatial audio data to obtain audio-visual driving control data.

7. A distributed audio processing system, characterized in that: The system includes: A transmission bus, used for receiving an audio processing task, and identifying a processing type of an audio code stream in the audio processing task according to a type field in a target audio transmission protocol, so as to transmit the audio code stream to a master node or a slave node according to the processing type, wherein the transmission bus performs data transmission based on a target audio transmission protocol, and the target audio transmission protocol supports transmission of different types of audio code streams and transmission modes; A master node, used for performing master node processing on the received audio code stream to obtain a processed audio code stream; At least one slave node is used to perform slave node processing on the received audio code stream to obtain a processed audio code stream, and transmit the processed audio code stream to at least one speaker for playback, wherein the master node and the slave node are connected using the transmission bus, and transmit the audio code stream and the processed audio code stream.

8. The system according to claim 7, characterized in that The master node is provided with a first audio subsystem supporting a spatial audio interface, and the first audio subsystem includes: A first input layer, used for receiving the audio code streams from different applications and converting the audio code streams into audio data packets in a unified format; A mixing layer, used for mixing and sorting the audio data packets transmitted by the first input layer to obtain mixed audio data packets; A first output layer, used for distributing the mixed audio data packets output by the mixing layer to the slave nodes or terminal nodes; The first configuration layer is used to configure the parameters and functions of the first input layer, the mixing layer and the first output layer according to different playback environments and user requirements.

9. The system according to claim 7, characterized in that A corresponding second audio subsystem is arranged on each of the slave nodes, and the second audio subsystem comprises: The second input layer is used to receive the audio code stream from the master node or other slave nodes, and convert the audio code stream into an audio data packet in a unified format; A processing layer, configured to perform slave node processing on the audio data packet transmitted by the second input layer to obtain a processed audio data packet; A second output layer, used for distributing the processed audio data packets output by the processing layer to other slave nodes or terminal nodes; The second configuration layer is used to configure the parameters and functions of the second input layer, the processing layer and the second output layer according to different playback environments and user requirements.

10. A distributed audio processing device, characterized in that: The device comprises: An acquisition module, configured to acquire an audio processing task received by a distributed audio processing system, wherein data is transmitted between at least two processing nodes in the distributed audio processing system based on a target audio transmission protocol; An identification module, used for identifying the processing type of the audio code stream in the audio processing task according to the type field in the target audio transmission protocol; A processing module is used to transmit the audio code stream to a target processing node corresponding to the processing type among the at least two processing nodes, so that the audio code stream is processed by the target processing node, and the processed audio code stream is transmitted to at least one speaker for playback.

11. An electronic device, characterized in that: include: A processor, and a memory connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 6.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 6 when executed by a processor.

13. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 6 when being executed by a processor.

Images