Audio data stream processing device and method and electronic equipment

Abstract

The invention relates to the field of data stream transmission processing, and particularly provides an audio data stream processing device and method and electronic equipment. The device comprises a plurality of audio processing modules, a back pressure driving module and a timing module, each of the plurality of audio processing modules is serially connected based on a preset interface and is used for processing initial audio data step by step and generating target audio data, and the timing module is connected with the back pressure driving module and is used for outputting the target audio data. The timing signal generation module is used for generating a timing signal with a fixed frequency based on an output sampling rate of target audio data, and the back pressure driving module is in serial connection with the last stage of audio processing module in the plurality of audio processing modules based on a preset interface and is used for responding to the timing signal and outputting one unit of target audio data, the preset interface is used for controlling asynchronous processing and asynchronous transmission of the audio data stream among the plurality of modules which are connected in series based on a preset rule under the same system clock. The overall efficiency and universality are improved through a unified interface and a timing module.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of data stream processing, and in particular to an audio data stream processing apparatus, method and electronic device. BACKGROUND

[0002] In an audio data stream processing system, multiple levels of audio algorithm processing modules are usually integrated to optimize the sound effect (such as gain adjustment, mixing and equalization, etc.). In addition, in order to adapt to different sampling rate specifications of various audio terminal devices, a sampling rate conversion module also needs to be integrated in the system. Therefore, the audio data needs to be processed and transmitted by multiple modules in turn.

[0003] At present, a global timing counter is used to keep the levels of modules capable of processing synchronously, which stipulates that after completing the internal data processing of each level of module, the data is sent to the next level of module at a fixed time under the control of a unified synchronization signal, that is, the data output rate of each level of module is completely consistent with the sampling rate. However, this scheme has the following problems: 1) High delay: the data output timing of each level of module is strictly limited, that is, the module must wait for the fixed synchronization time to transmit data even if it completes the processing in advance, resulting in data accumulation in front of each level of module, and the overall system delay is large and increases with the number of modules.

[0004] 2) Poor versatility: when the sampling rate changes, the timing logic of each level of module needs to be adjusted or redesigned, which is not self-adaptive and difficult to adapt to diversified audio terminal devices. SUMMARY

[0005] The present disclosure is proposed in view of the above problems. The present disclosure provides an audio data stream processing apparatus, method and electronic device.

[0006] According to one aspect of the present disclosure, an audio data stream processing apparatus is provided, the apparatus comprising: a plurality of audio processing modules, a backpressure driving module and a timing module, wherein each of the plurality of audio processing modules is serially connected based on a preset interface, for performing level-by-level processing on initial audio data to generate target audio data, the timing module is connected with the backpressure driving module, for generating a timing signal of a fixed frequency based on an output sampling rate of the target audio data, and the backpressure driving module is serially connected with a last level of audio processing module in the plurality of audio processing modules based on the preset interface, for outputting a unit amount of target audio data in response to the timing signal, wherein the preset interface is used to control asynchronous processing and asynchronous transmission of the audio data stream between the serially connected plurality of modules based on a preset backpressure rule under the same system clock.

[0007] In addition, the audio data stream processing device according to one aspect of the present disclosure, wherein the preset interface comprises: an input ready unit, an output data valid unit, an output ready unit, and an input data valid unit, wherein each of the plurality of modules connected in series is connected based on the input ready unit of the module and the output ready unit of the next stage module, and connected based on the output data valid unit of the module and the input data valid unit of the next stage module.

[0008] In addition, the audio data stream processing device according to one aspect of the present disclosure, wherein the input ready unit is configured to receive a ready signal sent by the next stage module; the output data valid unit is configured to send a data valid signal to the next stage module; the output ready unit is configured to send a ready signal to the previous stage module; and the input data valid unit is configured to receive a data valid signal sent by the previous stage module, wherein the ready signal indicates that the current module can receive one unit of audio data, the data valid signal indicates that the current module can send one unit of audio data, and the audio data is processed.

[0009] In addition, the audio data stream processing device according to one aspect of the present disclosure, wherein the preset back pressure rule comprises: when the current module is in a first state and receives a ready signal sent by the next stage module, sending a data valid signal to the next stage module; and when the current module sends one unit of audio data to the next stage module, sending a ready signal to the previous stage module.

[0010] In addition, the audio data stream processing device according to one aspect of the present disclosure, wherein the device further comprises: an acquisition module, a storage module, and an output module, wherein the acquisition module is connected to the storage module and configured to acquire initial audio data and send the initial audio data to the storage module; the storage module is connected in series to a first stage audio processing module in the plurality of audio processing modules based on the preset interface, and configured to store the initial audio data and send one unit of the initial audio data to the first stage audio processing module after receiving a ready signal; and the output module is connected to the back pressure driving module and configured to output target audio data to a terminal device.

[0011] In addition, the audio data stream processing device according to one aspect of the present disclosure, wherein the back pressure driving module is configured to include at least two units of target audio data.

[0012] In addition, the audio data stream processing device according to one aspect of the present disclosure, wherein the plurality of audio processing modules comprises: a sampling rate conversion module, and one or more of a gain module, a mix module, an equalization module, and a limiting module, wherein the sampling rate conversion module is configured to adjust the sampling rate of the audio data output by the previous stage module to meet the output sampling rate of the target audio data.

[0013] According to another aspect of the present disclosure, there is provided an audio data stream processing method applied to the audio data stream processing apparatus as described above, the method comprising: controlling, based on a preset back pressure rule, the audio data stream to be processed and transmitted asynchronously among a plurality of serially connected modules of the audio data stream processing apparatus under a same system clock, wherein the preset back pressure rule comprises: sending, when a current module is in a first state and a ready signal sent by a next stage module is received, a data valid signal to the next stage module; and sending, after the current module sends a unit amount of audio data to the next stage module, a ready signal to a previous stage module, wherein the ready signal indicates that the current module is capable of receiving a unit amount of audio data, the data valid signal indicates that the current module is capable of sending a unit amount of audio data, and the audio data is processed.

[0014] In addition, according to the audio data stream processing method of the present disclosure, the method further comprises: controlling the audio data stream to be output at a fixed frequency, wherein the fixed frequency is determined based on an output sampling rate of target audio data.

[0015] According to yet another aspect of the present disclosure, there is provided an electronic device comprising: a memory configured to store computer readable instructions; and a processor configured to execute the computer readable instructions to cause the electronic device to perform the audio data stream processing method as described above.

[0016] As will be described in detail below, the audio data stream processing apparatus according to the embodiments of the present disclosure decouples the processing rhythm of each stage of processing modules from the output sampling rate through a unified interface and a timing module, thereby improving the overall efficiency and the overall versatility and saving the hardware cost.

[0017] It is to be understood that both the foregoing general description and the following detailed description are exemplary and intended to provide further explanation of the subject technology. BRIEF DESCRIPTION OF DRAWINGS

[0018] The foregoing and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description, which proceeds with reference to the accompanying drawings. The drawings are provided to illustrate embodiments of the present disclosure and, together with the detailed description, serve to explain the present disclosure and do not constitute a limitation thereof. In the drawings, like reference numerals refer to like elements or steps throughout.

[0019] Figure 1 is a schematic block diagram illustrating an audio data stream processing apparatus according to an embodiment of the present disclosure.

[0020] Figure 2 is a schematic diagram illustrating a preset interface of an audio data stream processing apparatus according to an embodiment of the present disclosure.

[0021] Figure 3 This is a flowchart illustrating an audio data stream processing method according to an embodiment of the present disclosure.

[0022] Figure 4 This is a schematic diagram illustrating the data flow of an audio data stream processing apparatus according to an embodiment of the present disclosure.

[0023] Figure 5 This is a schematic diagram illustrating an audio data stream processing apparatus according to an embodiment of the present disclosure.

[0024] Figure 6 This is a timing diagram illustrating the data flow of a frequency data stream processing apparatus according to an embodiment of the present disclosure.

[0025] Figure 7 This is a hardware block diagram illustrating an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this disclosure more apparent, exemplary embodiments according to this disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments of this disclosure. It should be understood that this disclosure is not limited to the exemplary embodiments described herein.

[0027] Currently, audio data stream processing systems typically use a global timer / counter to constrain the data output timing of each processing module. In other words, the global timer / counter forcibly limits the output rhythm of each processing module in the audio processing chain. Each processing module can only transmit data together (output to the subsequent module) at a fixed time. Even if the current module has finished processing the data, it must wait for the unified instruction of the global timer / counter and cannot output to the subsequent module in advance.

[0028] 1) This practice of strictly limiting the timing of data output from each level of module forcibly binds the processing rhythm of each level of processing module to the system sampling rate, increases the waiting time of modules that finish processing first, causes data to accumulate in front of each level of module, resulting in a large overall system latency. Moreover, the more intermediate processing modules there are, the longer the latency becomes, causing high latency problems.

[0029] 2) When the audio sampling rate processed by this link changes (e.g., switching from the commonly used 48 kHz sampling rate to the 96 kHz or 192 kHz required by the Hi-Res audio certification standard), it forcibly speeds up the output rhythm of each processing module on the audio processing link. However, some slower processing modules have not yet completed their calculations, meaning their output is not ready when the synchronization signal arrives, leading to data loss or system errors. To avoid such errors, additional differentiated matching processing is required for these slower processing modules, resulting in increased costs and poor overall versatility.

[0030] 3) Although the Avalon Streaming Interface (Avalon-ST) bus protocol is suitable for the transmission of high-bandwidth, low-latency data streams, the protocol also needs to support multiple functions such as continuous transmission, packet processing, and delay settings. For audio data streams with fixed sampling rates and simple modes, this would result in functional redundancy and increased hardware overhead, leading to unnecessary waste.

[0031] To address the aforementioned problems, this disclosure provides an audio data stream processing apparatus, specifically as follows: Figures 1-6 describe.

[0032] Figure 1 This is a schematic block diagram illustrating an audio data stream processing apparatus according to an embodiment of the present disclosure. Figure 1 As shown, the audio data stream processing apparatus 1 of this disclosure embodiment may include at least: Multiple audio processing modules (audio processing module 11, ..., audio processing module 1n), reverse pressure drive module 21, and timing module 31.

[0033] Each of the multiple audio processing modules is based on a preset interface ( Figure 1 (Not shown) A serial connection is used to process the initial audio data step by step to generate target audio data; the timing module 31 is connected to the reverse pressure drive module 21 and is used to generate a fixed frequency timing signal based on the output sampling rate of the target audio data; the reverse pressure drive module 21 is serially connected to the last-stage audio processing module 1n among multiple audio processing modules based on a preset interface, and is used to output a unit amount of target audio data in response to the timing signal. Specifically, the preset interface is used to control the asynchronous processing and asynchronous transmission of audio data streams among multiple serially connected modules under the same system clock based on preset reverse pressure rules.

[0034] like Figure 1As shown, the audio processing link includes at least serially connected audio processing modules 11, ..., 1n and a reverse pressure drive module 21. Any two adjacent modules on this link are connected via a unified interface (i.e., a preset interface), which operates based on preset reverse pressure rules. A timing module 31 is configured at the end of the link, connected only to the reverse pressure drive module 21, and its interface differs from the preset interface. A detailed description of the preset interface will be provided in [reference needed]. Figure 2 .

[0035] Specifically, a timing module 31 is used to replace the global timer / counter, and the timing module 31 only acts on the back pressure drive module 21, that is, it only controls the rhythm of the final output (output sampling rate) and no longer controls the processing rhythm of other modules in the audio processing link (such as audio processing module 11, ..., audio processing module 1n). These modules no longer need to receive a unified synchronization signal, but instead realize the flow of audio data stream based on their own processing status and the needs of the subsequent modules (i.e., preset back pressure rules).

[0036] In other words, the audio data stream processing device of this embodiment decouples the processing rhythm of each processing module from the output sampling rate. All other modules in the audio processing link, except for the reverse pressure drive module 21, do not need to know the output sampling rate. They only need to respond to the reverse pressure signal of the subsequent module. This greatly reduces the waiting time of each module, reduces latency, and improves overall efficiency.

[0037] Meanwhile, since the timing module 31 of the audio data stream processing device in this embodiment only operates on the reverse pressure drive module 21, only the reverse pressure drive module 21 knows when the output sampling rate changes (e.g., speeds up). Other modules on the link only notice that the frequency of the reverse pressure signal sent by the downstream module has increased, and their own processing will also speed up, thereby driving all levels of modules in the entire link to speed up the processing rhythm. There will be no need to perform additional differentiated processing on some modules because some modules are slower, thus improving the overall versatility.

[0038] Furthermore, the preset interface of the audio data stream processing device in this embodiment operates based on preset backpressure rules, which is more suitable for audio data streams with fixed sampling rates and simple patterns, saving logic resources and hardware costs.

[0039] Figure 2 This is a schematic diagram illustrating a preset interface of an audio data stream processing apparatus according to an embodiment of the present disclosure. Figure 2 As shown, the preset interface includes: input ready unit a, output data valid unit b, output ready unit c, and input data valid unit d.

[0040] like Figure 2As shown, the current module can be any module in the aforementioned audio processing link that is serially connected based on a preset interface, and it is connected to the front-end module and the back-end module based on the preset interface. Figure 2 (Not shown in the diagram). Specifically, the current module is connected to the output ready unit c of the subsequent module based on its own input ready unit a, and is connected to the input data valid unit d of the subsequent module based on its own output data valid unit b. Similarly, the current module is connected to the input ready unit a of the previous module based on its own output ready unit c, and is connected to the output data valid unit b of the previous module based on its own input data valid unit d.

[0041] The ready units (a and c) are used to transmit ready signals, and the data valid units (b and d) are used to transmit data valid signals. Specifically, the input ready unit a of the current module is used to receive the ready signal sent by the subsequent module, where the ready signal indicates that it can receive one unit of audio data. That is, when the subsequent module needs to request data from the current module (i.e., when its buffer has at least one unit), it will send a ready signal to receive one unit of audio data for processing.

[0042] The current module's output data valid unit b is used to send a data valid signal to the subsequent module. This data valid signal indicates that a unit of audio data can be sent, and that the audio data is processed. In other words, when the current module receives a ready signal from the subsequent module, if it has already completed processing, it will send a data valid signal to the subsequent module, informing it that the data is valid (or processed and reliable) and has been placed on the output data bus.

[0043] Similarly, the current module's output ready unit c is used to send a ready signal to the preceding module; the input data valid unit d is used to receive the data valid signal sent by the preceding module. In other words, when the current module requests data, it sends a ready signal to the preceding module; after receiving this ready signal, if the preceding module has already processed the data, it sends a data valid signal to the current module, informing it that it has finished processing the data and that the data is valid (or reliable) and placed on the output data bus. See below for further details. Figure 3 and Figure 4 The description continues how the audio data stream flows between multiple modules connected in a serial manner.

[0044] Figure 3 This is a flowchart illustrating an audio data stream processing method according to an embodiment of the present disclosure. Specifically, the audio data stream processing method may include the following steps.

[0045] In step S301, based on preset rules, the audio data stream is controlled to be processed and transmitted asynchronously among multiple serially connected modules of the audio data stream processing device under the same system clock. The preset rules include: when the current module is in the first state and receives a ready signal sent by the downstream module, it sends a data valid signal to the downstream module; when the current module sends a unit amount of audio data to the downstream module, it sends a ready signal to the upstream module. The ready signal indicates that the current module can receive a unit amount of audio data, and the data valid signal indicates that the current module can send a unit amount of audio data, and the audio data is processed.

[0046] Figure 4 This is a schematic diagram illustrating the data flow of an audio data stream processing apparatus according to an embodiment of the present disclosure. Figure 4 As shown, the audio processing link consists of three serially connected modules (backpressure drive module 21, audio processing module 1n, and audio processing module 2n). The above audio data stream processing method will be described in detail using an example.

[0047] Specifically, the reverse pressure drive module 21 can be understood as a subsequent module, and the audio processing module 1n is the current module. This is the front-end module. When the reverse pressure drive module 21 receives the timing signal sent by the timing module 31, it means that the reverse pressure drive module 21 needs to output a unit amount of target audio data. Therefore, the internal buffer of the reverse pressure drive module 21 is decremented by 1 (i.e., a unit amount of buffer is freed up). Then, the reverse pressure drive module 21 sends a ready signal to the audio processing module 1n to request data. The audio processing module 1n will check whether its own state meets the first state. The first state is that the internal processing calculation is completed. That is to say, only when the internal processing calculation of the audio processing module 1n is completed and the ready signal sent by the reverse pressure drive module 21 is received, and both of these occur simultaneously, will the audio processing module 1n send a data valid signal to the reverse pressure drive module 21 based on the output data valid unit b.

[0048] When the audio processing module 1n sends a unit amount of audio data to the reverse pressure drive module 21, its internal buffer is decremented by 1. Then, when the audio processing module 1n sends a signal to the audio processing module 21... Send a ready signal to request data; audio processing module It will check whether its own state meets the first state. When it meets the first state, the audio processing module... A data valid signal is sent to the audio processing module 1n, and the audio processing module 1n receives the data valid signal through its own input data valid unit d.

[0049] Similarly, other modules connected in series on the audio processing link request data step by step based on the same preset backpressure rules, so as to realize the flow of audio data streams throughout the entire link.

[0050] It should be noted that if the current module is in the second state, even if it receives a ready signal from the subsequent module, the current module will not send a data valid signal to the subsequent module. The second state indicates that the module is in the process of processing (i.e., the processing has not yet been completed).

[0051] This concludes the description of the audio data stream flow based on preset backpressure rules. In addition to the audio data stream processing apparatus 1 of this embodiment, Figure 1 In addition to the modules described above, it may also include: an acquisition module, a storage module, and an output module.

[0052] The acquisition module is connected to the storage module and is used to acquire initial audio data and send it to the storage module. The storage module is serially connected to the first-level audio processing module 11 among multiple audio processing modules based on a preset interface. It is used to store the initial audio data and, upon receiving a ready signal, to send a unit amount of initial audio data to the first-level audio processing module 11. The output module is connected to the reverse pressure drive module 21 and is used to output target audio data to the terminal device. The audio data stream processing apparatus 1 of this disclosure will be further described in detail below with reference to a specific example embodiment.

[0053] Figure 5 This is a schematic diagram illustrating an audio data stream processing apparatus according to an embodiment of the present disclosure. Figure 5 As shown, the audio data stream processing device 1 includes: a digital signal processor 101, a first-in-first-out memory 102, a gain module 103, a mixing module 104, an equalization module 105, a limiting module 106, a transsampling module 107, a reverse voltage drive source 108, a sampling rate timer 109, and an output interface 110.

[0054] The first-in-first-out (FIFO) memory 102 to the reverse-voltage drive source 108 are all serially connected based on a preset interface. The input ready unit a, output data valid unit b, output ready unit c, and input data valid unit d in the preset interface can be understood as physical signal lines. These signal lines carry a voltage level, specifically represented by logic values ​​1 or 0 to indicate a high or low level. The output interface 110 can be further connected to an audio terminal device.

[0055] Specifically, the digital signal processor 101 can be understood as the aforementioned acquisition module, the first-in-first-out memory 102 can be understood as the aforementioned storage module, the gain module 103 to the sampling module 107 can be understood as multiple audio processing modules (audio processing module 11, ..., audio processing module 1n), the reverse pressure drive source 108 can be understood as the reverse pressure drive module 21, the sampling rate timer 109 can be understood as the timing module 31, and the output interface 110 can be understood as the output module.

[0056] The digital signal processor 101 acquires the initial audio data and writes it into the first-in-first-out memory 102. Each stage of the gain module 103 to the limiting module 106 performs different processing on the initial audio data to generate the processed audio data of each stage. Finally, the resampling module 107 performs resampling processing on the audio data processed by the limiting module 106 according to the output sampling rate of the target audio data to obtain the target audio data.

[0057] The sampling rate timer 109 determines the frequency of the data acquisition timing signal based on the output sampling rate. In other words, the frequency of the timing signal depends on the output sampling rate. Different sampling rates correspond to different frequency timing signals, and the frequency of the timing signal is the same as the output data sampling rate.

[0058] In one embodiment, it is assumed that the operating clock frequency of the sampling rate timer 109 is... = 24.576MHz, output sampling rate = 16KHz, then the total number of counts that the sampling rate timer 109 needs to count within one sampling period is:

[0059] In other words, the sampling rate timer 109 internally starts counting from 0. When it counts to 1535 (i.e., 1536-1), it outputs a pulse signal (i.e., a timing signal) to the reverse voltage drive source 108 to indicate that it outputs one unit of target audio data. Then, it continues to count in a loop, sending a timing signal every time it counts from 0 to 1535. That is, the counting range is 0 to N-1. When it counts to N-1, it sends a high-level pulse timing signal with a pulse width of one clock cycle.

[0060] In another embodiment, if the operating clock frequency of the sampling rate timer 109 is set... =24.576MHz, set the output data sampling rate = 96KHz, then the total number of counts that the sampling rate timer 109 needs to count in one sampling period is:

[0061] That is, after the sampling rate timer 109 counts from 0 to 255, it indicates that a unit amount of target audio data will be output to the reverse voltage drive source 108.

[0062] The reverse pressure drive source 108 receives a timing signal and, based on the state of its internal buffer data, outputs a unit amount of target audio data to the output interface 110, while simultaneously outputting a ready signal to the transsampling module 107. The reverse pressure drive source 108 can be configured to include at least two units of target audio data.

[0063] In one embodiment, the reverse-voltage drive source 108 has a buffer capacity of two units to save hardware resources. As long as its internal buffer is not full, it sends a ready signal (i.e., pulls up the output ready unit c level) to the preceding pre-sampling module 107, indicating that it can receive new valid data (i.e., data processed by the pre-sampling module 107). In other words, there is almost no data buffering between multi-level modules (only a very small buffer of two units), further reducing hardware overhead.

[0064] It should be noted that when the reverse pressure drive source 108 is filled with its internal buffer capacity for the first time, it will start to respond to the timing signal issued by the sampling rate timer 109 and start to output target audio data to the output interface 110. Each time a timing signal is received, one unit of target audio data will be output, thereby realizing the output of fixed sampling rate audio data from the output interface 110.

[0065] It is understood that the output interface 110 may include an integrated circuit built-in audio bus (Integrated Interchip Sound, IIS), a codec (Coder-Decoder, CODEC), etc., without any specific limitations, all of which are within the protection scope of this disclosure.

[0066] The subsampling module 107 is used to adjust the sampling rate of the audio data output by the preamplifier module to meet the output sampling rate of the target audio data (for example, the subsampling module 107 can perform upsampling output, such as receiving 48KHz and outputting 96KHz; similarly, the subsampling module 107 can also perform downsampling output, such as receiving 48KHz and outputting 16KHz). When its internal processing calculation is completed and it receives a ready signal from the reverse voltage drive source 108 (i.e., its own input ready unit a is pulled high), it pulls up the level of its own output data valid unit b and outputs a unit amount of target audio data to the reverse voltage drive source 108.

[0067] After the sampling module 107 outputs a unit of target audio data, its internal buffer is decremented by 1, and its input ready unit a level is raised to request data from the limiting module 106. This process continues until the data is transmitted to the first-in-first-out memory 102.

[0068] In one embodiment, the digital signal processor 101 can be a dedicated audio digital signal processor (DSP) core, such as a Cadence HiFi DSP, or other control processors, such as an application processor central processing unit (APCPU). It can read data (i.e., initial audio data) from the audio source from the Double Data Rate Synchronous Dynamic Random Access Memory (DDR memory) via Direct Memory Access (DMA) and control the writing to the first-in-first-out memory 102 cache when it is not full.

[0069] When the first-in-first-out memory 102 has data cached inside (i.e., not empty) and receives a ready signal from the gain module 103, the first-in-first-out memory 102 reads out the data and simultaneously pulls up its own output data valid unit b, outputting the initial audio data to the gain module 103.

[0070] It should be noted that the digital signal processor 101 to the output interface 110 in this disclosure are merely illustrative examples. Furthermore, the gain module 103 to the transsampling module 107 can be flexibly added, trimmed, or rearranged in order, and this does not constitute a limitation. The following will combine... Figure 6 The changes in the voltage levels of each unit within the module and its preset interfaces are explained.

[0071] Figure 6 This is a timing diagram illustrating the data flow of a frequency data stream processing apparatus according to an embodiment of the present disclosure. Figure 6 As shown, clk represents a periodic square wave, which is the time base for all actions. The rising edge triggers the update of all signal states, which can be understood as a metronome for data processing and signal interaction.

[0072] in_ready represents the input ready unit a. A high level in_ready indicates that the subsequent module is requesting data from the current module; a low level in_ready indicates that the subsequent module does not currently have a request.

[0073] `in_valid` represents the valid input data unit `d`. A high level in `in_valid` indicates that the preceding module has sent a single unit of valid data to the current module, lasting only one clock cycle, to conform to the discontinuous transmission characteristic of audio data. It's important to note that when `in_valid` goes high, the current module immediately initiates internal processing (triggers a change in `cal_flag`) and sets `out_ready` to 0, indicating that the preceding module has no immediate data request and is processing the current data.

[0074] cal_flag indicates the current module's internal processing. A high level in cal_flag indicates that the current module is processing data (such as performing algorithms for gain, mixing, and sample rate conversion). When invalid is pulled high, cal_flag is immediately set to 1 and can remain high for multiple clk cycles (the duration of which is the time required for the module's internal processing) until the processing is completed and then pulled low.

[0075] `cal_done` indicates that the current module's internal processing is complete. A high level for `cal_done` (the first state) indicates that the current module has finished data processing and can send data to subsequent stages. Specifically, when `cal_flag` is pulled low (processing finished), `cal_done` is immediately set to 1. If `in_ready` is high at this time (i.e., the subsequent stage is ready), then `cal_done` will be pulled low after `out_valid` goes high (i.e., data has been sent).

[0076] out_ready represents the output ready unit c. Specifically, the state of out_ready is controlled by the linkage between in_valid and out_valid: when in_valid goes high, out_ready is set to 0 on the rising edge of clk (informing the preceding stage "no data request is currently being processed"); when out_valid goes high, out_ready is set to 1 on the rising edge of clk (informing the preceding stage "data has been sent and new data can be received").

[0077] out_valid represents the valid unit b of output data. Specifically, out_valid is only pulled high when in_ready=1 and cal_done=1, and lasts for only 1 clk cycle (ensuring accurate transmission of one unit of audio data).

[0078] The following will be based on Figure 6 As shown, the data flow process is further described.

[0079] Phase 1: In this phase, only out_ready is at a high level (i.e., logic value 1) while all others are at a logic value of 0. This indicates that the current module is requesting data from the previous module, but is in a state of no data input, no data processing, and no data output.

[0080] In the second stage, the current module's in_ready level goes high, indicating that the subsequent module is requesting data from the current module. At this stage, the current module's out_ready level is still high, indicating that the previous module has not yet transmitted the processed data to the current module. All other logic values ​​are 0, indicating that there is no data flow.

[0081] Phase 3: The current module's `invalid` level goes high (for only one clock cycle), indicating that the preceding module has sent a unit amount of valid data to the current module. Subsequently, the `out_ready` level drops to 0, indicating that the current module, after receiving the aforementioned unit amount of valid data, will not request new data further; simultaneously, the `cal_flag` level goes high, indicating that the current module has begun processing the received data; the current module's `in_ready` level remains high, indicating that no data has yet been sent to the subsequent module; all other logic values ​​are 0.

[0082] Phase 4: The current module's `cal_flag` is high (for several clock cycles), indicating that the current module is continuously processing data sent by other modules. When `cal_flag` is set to 0, `cal_done` is momentarily pulled high, indicating that the current module has finished processing. Since `in_ready` has been set for more than one clock cycle, it means that subsequent modules have been requesting data. Now that the current module has finished processing, `out_valid` is pulled high, preparing to output data to subsequent modules. All other logic values ​​are 0.

[0083] Phase 5: The current module's `out_valid` is high (for only one clock cycle), indicating that the current module is sending a single unit of valid data to the next-level module. The current module's internal buffer is decremented by 1, so after sending the data, the `out_ready` level is pulled high to request data from the next-level module. Simultaneously, after the current module finishes sending data, `cal_done` is also set to 0; and because the next-level module has received the data, the current module's `in_ready` is set to 0, meaning the next-level module has no further requests.

[0084] Phase 6: In the current module, only out_ready is at a high level, while the rest are set to 0. This indicates that the current module is waiting for the preceding module to send data in order to enter the next processing cycle.

[0085] It should be noted that the above are all exemplary descriptions to more clearly illustrate the workflow, and the data flow rhythm of the audio data stream processing device disclosed herein is not limited thereto.

[0086] Figure 7 This is a hardware block diagram illustrating an electronic device 700 according to an embodiment of the present disclosure. The electronic device according to an embodiment of the present disclosure includes at least a processor and a memory for storing computer-readable instructions. When the computer-readable instructions are loaded and executed by the processor, the processor performs the audio data stream processing method as described above.

[0087] Figure 7 The illustrated electronic device 700 specifically includes a central processing unit (CPU) 701, a graphics processing unit (GPU) 702, and a memory 703. These units are interconnected via a bus 704. The CPU 701 and / or GPU 702 can function as the aforementioned processors, and the main memory 703 can function as the aforementioned memory storing computer-readable instructions. Furthermore, the electronic device 700 may also include a communication unit 705, a storage unit 706, an output unit 707, an input unit 708, and an external device 709, all of which are also connected to the bus 704.

[0088] The audio data stream processing apparatus, method, and electronic device according to embodiments of the present disclosure have been described above with reference to the accompanying drawings. The audio data stream processing apparatus according to embodiments of the present disclosure decouples the processing rhythm of each processing module from the output sampling rate through a unified interface and a timing module, thereby improving overall efficiency, enhancing overall versatility, and saving hardware costs.

[0089] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0090] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0091] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0092] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0093] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0094] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0095] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0096] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. An audio data stream processing device, characterized in that, The device includes: Multiple audio processing modules, reverse pressure drive module, and timing module. Each of the plurality of audio processing modules is connected serially based on a preset interface, and is used to process the initial audio data step by step to generate the target audio data. The timing module is connected to the reverse pressure drive module and is used to generate a fixed-frequency timing signal based on the output sampling rate of the target audio data. The reverse pressure drive module is serially connected to the last-stage audio processing module among the plurality of audio processing modules based on the preset interface, and is used to output a unit amount of the target audio data in response to the timing signal. The preset interface is used to control the asynchronous processing and asynchronous transmission of audio data streams among multiple serially connected modules under the same system clock, based on preset backpressure rules.

2. The audio data stream processing apparatus as described in claim 1, characterized in that, The preset interface includes: Input ready unit, output data valid unit, output ready unit, input data valid unit. Each of the serially connected modules is connected to the output ready unit of its subsequent module based on its own input ready unit, and is connected to the input data valid unit of the subsequent module based on its own output data valid unit.

3. The audio data stream processing apparatus as described in claim 2, characterized in that, The input ready unit is used to receive the ready signal sent by the subsequent module; The output data valid unit is used to send a data valid signal to the subsequent module; The output ready unit is used to send the ready signal to the preceding module; The input data validation unit is used to receive the data validation signal sent by the front-end module. The ready signal indicates that the current module is capable of receiving one unit of audio data. The data validity signal indicates that the current module is capable of sending a unit amount of audio data, and that the audio data is processed.

4. The audio data stream processing apparatus as described in claim 3, characterized in that, The preset back pressure rules include: When the current module is in the first state and receives the ready signal sent by the downstream module, the data valid signal is sent to the downstream module; After the current module sends a unit amount of audio data to the downstream module, it sends the ready signal to the upstream module.

5. The audio data stream processing apparatus as described in claim 4, characterized in that, The device further includes: Acquisition module, storage module, output module The acquisition module is connected to the storage module and is used to acquire the initial audio data and send it to the storage module. The storage module is serially connected to the first-level audio processing module among the plurality of audio processing modules based on the preset interface, and is used to store the initial audio data, and to send a unit amount of the initial audio data to the first-level audio processing module after receiving the ready signal; The output module is connected to the reverse pressure drive module and is used to output the target audio data to the terminal device.

6. The audio data stream processing apparatus as described in claim 1, characterized in that, The reverse pressure drive module is configured to include at least two units of the target audio data.

7. The audio data stream processing apparatus as described in claim 1, characterized in that, The plurality of audio processing modules include: The sampling rate conversion module, and one or more of the following modules: gain module, mixing module, equalization module, and limiting module. The sampling rate conversion module is used to adjust the sampling rate of the audio data output by the preceding module so that it meets the output sampling rate of the target audio data.

8. An audio data stream processing method, characterized in that, Applied to the audio data stream processing apparatus as described in any one of claims 1-7, the method comprises: Based on preset backpressure rules, the audio data stream is controlled to be processed and transmitted asynchronously among multiple serially connected modules of the audio data stream processing device under the same system clock. The preset backpressure rules include: when the current module is in the first state and receives a ready signal from the downstream module, it sends a data valid signal to the downstream module; when the current module sends one unit of audio data to the downstream module, it sends the ready signal to the upstream module. The ready signal indicates that the current module can receive one unit of audio data, and the data valid signal indicates that the current module can send one unit of audio data, wherein the audio data is processed.

9. The audio data stream processing method as described in claim 8, characterized in that, The method further includes: The audio data stream is controlled to be output at a fixed frequency, wherein the fixed frequency is determined based on the output sampling rate of the target audio data.

10. An electronic device, characterized in that, include: Memory, used to store computer-readable instructions; as well as A processor for executing the computer-readable instructions, causing the electronic device to perform the audio data stream processing method as described in claim 8 or 9.

Images