An audio signal data cooperative processing method, system, device and medium

Abstract

The application discloses an audio signal data cooperative processing method, system, device and medium, and relates to the technical field of data processing.The method comprises the following steps: obtaining periodic cooperative basic data and a last processing period state, and generating a processing period cooperative state code; obtaining an audio link load adjustment amount according to the number of effective channels, the number of enabled sound effect processing units, the number of output buffer playable samples, the number of buffer coverage shortage rising times and the number of input buffer out-of-bound times, and determining a processing end resource execution gear in combination with the processing period cooperative state code; determining a periodic input buffer sample budget according to the resource gear, converting the periodic input buffer sample budget into a buffer coverage duration, and comparing the buffer coverage duration with an actual processing delay to obtain a buffer coverage shortage duration; and accordingly executing normal, mild shortage or severe shortage processing.The application is used for processing the periodic mismatch problem between the collection, processing and playing output of the same audio link, and reducing the output discontinuity risk caused by the buffer coverage shortage.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and specifically to a method, system, device, and medium for collaborative processing of audio signal data. Background Technology

[0002] In portable audio devices, real-time communication terminals, mobile live streaming devices, conferencing terminals, and multimedia devices with local audio processing capabilities, the audio signal processing chain typically includes continuous steps such as audio acquisition, input buffer writing, digital signal processing, output buffer storage, and playback output. As audio processing scenarios expand from single voice acquisition to complex processing tasks such as multi-channel mixing, echo suppression, noise reduction, dynamic range control, equalization processing, and latency compensation, the computational load on the processing end changes in different processing cycles, thereby affecting the input buffer consumption rate and the output buffer replenishment rate.

[0003] Existing audio processing solutions typically operate the acquisition, processing, and playback output ends according to fixed strategies. For example, the acquisition end continuously writes audio samples at a base sampling rate, the processing end performs digital signal processing according to a fixed thread budget or fixed resource level, and the playback output end outputs audio samples according to a fixed buffer reading strategy. When the number of active channels increases, the number of activated audio effect processing units increases, or the actual processing latency of the processing end increases, the fixed resource configuration cannot reflect the execution results of the previous processing cycle in a timely manner, nor can it adjust the input buffer budget in a timely manner based on the output buffer level of the playback output end.

[0004] While some solutions can adjust cache size based on processing latency or cache level, they typically rely on only a single latency or cache occupancy data point, lacking a combined assessment of the current main load source, historical cache coverage insufficiency trends, and input cache out-of-bounds trends. Furthermore, the input cache sample budget represents the number of samples, while the actual processing latency represents the duration. If the input cache sample budget is not converted into cache coverage time based on the base sampling rate, and control is performed solely based on cache quantity and processing latency, the judgment boundaries may be unclear, affecting the assessment of cache coverage insufficiency in the current processing cycle.

[0005] Furthermore, if a short-term increase in latency occurs at the processing end, stopping the acquisition or discarding audio frames of the current processing cycle may affect the continuity of audio output. If a continuous backlog occurs at the processing end, continuing to write new audio samples to the input buffer will increase the processing end's emptying burden and may cause the output buffer level to continue to drop. Therefore, a collaborative processing method is needed for the same audio signal processing link, enabling the acquisition end, processing end, and playback output end to perform periodic-level linkage based on the status of the previous processing cycle, the current processing cycle load, the output buffer level, and the actual processing latency. Summary of the Invention

[0006] To address the technical problems existing in the background art mentioned above, the present invention provides an audio signal data collaborative processing method, system, device and medium, which solves the problems of insufficient buffer coverage and decreased output continuity caused by periodic load changes between acquisition, processing and playback output within the same audio signal processing link.

[0007] An audio signal data collaborative processing method includes: acquiring periodic collaborative basic data of the same audio signal processing link, as well as the audio link load level, delay status flag, output buffer level, and resource level execution result of the previous processing cycle, and generating a processing cycle collaborative status code for the current processing cycle; acquiring the audio link load adjustment amount based on the number of effective channel occupancy, the number of activated audio effect processing units, the number of playable samples in the output buffer, the number of buffer coverage insufficiency rises, and the number of input buffer out-of-bounds rises, and determining the processing end resource execution level in conjunction with the processing cycle collaborative status code; determining the periodic input buffer sample budget based on the audio link load adjustment amount, the processing end resource execution level, and the output buffer level, converting the periodic input buffer sample budget into buffer coverage duration according to the basic sampling rate, and acquiring the buffer coverage insufficiency duration based on the buffer coverage duration and the actual processing delay; performing normal processing, mild buffer coverage insufficiency processing, or severe buffer coverage insufficiency processing based on the buffer coverage insufficiency duration, and resuming the processing cycle scheduling after meeting the recovery conditions.

[0008] Optionally, generating the processing cycle collaborative status code for the current processing cycle includes: reading the same initial link configuration data, the current processing cycle number, the audio link load level of the previous processing cycle, the delay status flag of the previous processing cycle, the output buffer level of the previous processing cycle, and the resource level execution result of the previous processing cycle; concatenating the initial link configuration data, the current processing cycle number, the audio link load level of the previous processing cycle, the delay status flag of the previous processing cycle, the output buffer level of the previous processing cycle, and the resource level execution result of the previous processing cycle into status input data according to the same field order; encoding the audio link load level of the previous processing cycle, the delay status flag of the previous processing cycle, the output buffer level of the previous processing cycle, and the resource level execution result of the previous processing cycle in the status input data to obtain status fields; performing a deterministic verification operation on the status input data to obtain verification fields; combining the status fields and verification fields according to a preset field order and mapping them to the value range of the processing cycle collaborative status code to obtain the processing cycle collaborative status code for the current processing cycle.

[0009] Optionally, obtaining the audio link load adjustment includes: obtaining the channel occupancy ratio based on the number of active channels and the maximum number of active channels; obtaining the audio effect processing unit occupancy ratio based on the number of enabled audio effect processing units and the maximum number of audio effect processing units; obtaining the output buffer level insufficiency based on the number of playable samples in the output buffer and the target playback level; obtaining the buffer coverage insufficiency increase ratio based on the number of times the buffer coverage insufficiency duration has increased relative to the previous processing cycle in the most recent confirmed processing cycles; obtaining the input buffer out-of-bounds ratio based on the number of times the input buffer occupancy has exceeded the input buffer occupancy boundary in the most recent confirmed processing cycles; selecting the largest among the channel occupancy ratio, audio effect processing unit occupancy ratio, and output buffer level insufficiency as the main load value, and obtaining the audio link load adjustment amount based on the main load value, the buffer coverage insufficiency increase ratio, and the input buffer out-of-bounds ratio.

[0010] Optionally, determining the resource execution level at the processing end includes: the processing end reads the delay status flag of the previous processing cycle based on the processing cycle coordination status code, and prioritizes the determination according to the order of third resource execution level, second resource execution level, and first resource execution level. When the audio link load adjustment is within the third range, or when a severe buffer overlay occurred in the previous processing cycle and the recovery conditions have been met, the processing end is determined to be at the third resource execution level. When the conditions for the third resource execution level are not met, and the audio link load adjustment is within the second range, or when a mild buffer overlay occurred in the previous processing cycle, the processing end is determined to be at the second resource execution level. When the conditions for both the third and second resource execution levels are not met, and the audio link load adjustment is within the first range and no buffer overlay occurred in the previous processing cycle, the processing end is determined to be at the first resource execution level. If a severe cache under-coverage state occurs in the previous processing cycle and the recovery conditions are not met, the current processing cycle will continue to execute the severe cache under-coverage processing and will not resume the normal processing cycle scheduling; among them, the processing end resource execution level is used to determine the processing thread budget, the audio effect processing unit execution budget, the data moving priority, and the upper limit of the working frequency.

[0011] Optionally, determining the periodic input buffer sample budget includes: when the processing end resource execution level is the first resource execution level and the output buffer level is the second or third level, determining the periodic input buffer sample budget based on the baseline periodic input buffer sample budget; when the processing end resource execution level is the second resource execution level, determining the periodic input buffer sample budget based on the baseline periodic input buffer sample budget and the sample increment corresponding to the audio link load adjustment; when the processing end resource execution level is the third resource execution level, or the output buffer level is the first level, increasing the periodic input buffer sample budget; when the output buffer level is the third level, limiting the growth rate of the periodic input buffer sample budget; wherein, the periodic input buffer sample budget is not lower than the baseline periodic input buffer sample budget and not higher than the maximum periodic input buffer sample budget.

[0012] Optionally, obtaining the insufficient buffer coverage duration includes: converting the periodic input buffer sample budget into a buffer coverage duration based on the periodic input buffer sample budget and the base sampling rate; recording the input completion time when the audio frame corresponding to the current processing cycle is written to the input buffer; recording the output completion time when the audio frame completes digital signal processing and is written to the output buffer; obtaining the actual processing delay based on the time interval between the output completion time and the input completion time; comparing the actual processing delay with the buffer coverage duration on the same scale, and determining the time when the actual processing delay exceeds the buffer coverage duration as the insufficient buffer coverage duration; when the actual processing delay does not exceed the buffer coverage duration, determining the insufficient buffer coverage duration as zero.

[0013] Optionally, the processing based on the duration of insufficient cache coverage—whether it's normal processing, mild insufficient cache coverage processing, or severe insufficient cache coverage processing—includes the following: when the duration of insufficient cache coverage does not exceed the first insufficient boundary, write audio samples according to the current processing cycle's input cache sample budget, complete digital signal processing according to the processing end resource execution level of the current processing cycle, and record the delay status flag as normal; when the duration of insufficient cache coverage exceeds the first insufficient boundary but does not exceed the second insufficient boundary, stop writing new audio samples to the input cache area of ​​the current processing cycle, transfer subsequently acquired audio samples to the input cache area or temporary holding area corresponding to the next processing cycle, transfer unprocessed audio frames to the waiting queue of the next processing cycle, and output the previous processing cycle. The system executes short-term hold frames or spare audio frames at the end of the input buffer and records the delay status as a mild buffer undercoverage state. When the buffer undercoverage duration exceeds the second undercoverage boundary, it pauses writing new audio samples to the input buffer, prioritizes emptying audio frames already in the input buffer, pauses receiving new input buffer write requests, and detects the computational utilization, input buffer utilization, output buffer level, processing thread blocking status, and memory usage status. It outputs a protection audio frame and records the delay status as a severe buffer undercoverage state. The recovery conditions include the computational utilization being lower than the computational utilization boundary, the input buffer utilization being lower than the input buffer utilization boundary, the output buffer level being the second or third level, and no processing thread blocking status being detected.

[0014] An audio signal data collaborative processing system is also provided, comprising: a periodic status generation module, used to acquire periodic collaborative basic data of the same audio signal processing link, as well as the audio link load level, delay status flag, output buffer level, and resource level execution result of the previous processing cycle, and generate a processing cycle collaborative status code for the current processing cycle; a resource level determination module, used to acquire the audio link load adjustment amount based on the number of effective channel occupancy, the number of enabled audio effect processing units, the number of playable samples in the output buffer, the number of times the buffer coverage is insufficient and the number of times the input buffer exceeds the limit, and to determine the processing end resource execution level in combination with the processing cycle collaborative status code; a buffer budget determination module, used to determine the periodic input buffer sample budget based on the audio link load adjustment amount, the processing end resource execution level, and the output buffer level, convert the periodic input buffer sample budget into buffer coverage duration according to the basic sampling rate, and acquire the buffer coverage insufficient duration based on the buffer coverage duration and the actual processing delay; and a hierarchical link protection module, used to perform normal processing, mild buffer coverage insufficient processing, or severe buffer coverage insufficient processing based on the buffer coverage insufficient duration, and to restore the processing cycle scheduling after the recovery conditions are met.

[0015] An electronic device is also provided, comprising: a memory having a computer program stored thereon; and a processor for executing the computer program in the memory for implementing an audio signal data collaborative processing method.

[0016] A non-transitory computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements a method for collaborative processing of audio signal data.

[0017] The beneficial effects of this invention are reflected in: In the entire audio signal data collaborative processing method, firstly, the audio link load level, latency status flag, output buffer level, and resource allocation execution results of the previous processing cycle are inherited through the processing cycle collaborative status code. This allows the resource configuration and buffer budget of the current processing cycle to refer to the confirmed link operation status of the previous processing cycle, rather than adjusting solely based on the current instantaneous data. Furthermore, the audio link load adjustment amount is obtained based on the number of effective channel occupancy, the number of activated audio effect processing units, the number of playable samples in the output buffer, the number of buffer coverage insufficiency increases, and the number of input buffer out-of-bounds increases. This ensures that the load adjustment amount includes the main load sources of the current processing cycle, the output buffer insufficiency situation, and the mismatch trend within the recent confirmed processing cycles. Further, the periodic input buffer sample budget is converted into buffer coverage time according to the basic sampling rate and then compared with the actual processing latency at the processing end using the same dimensions. This allows for determination of whether the input buffer sample budget covers the actual processing time at the processing end and reduces the ambiguity of control boundaries caused by directly comparing sample quantity and time length. Furthermore, based on the duration of insufficient cache coverage, normal processing, mild insufficient cache coverage processing, or severe insufficient cache coverage processing are performed. When there is mild insufficient cache coverage, input cache write transfer, pending queue transfer, and short-term hold frame output are performed. When there is severe insufficient cache coverage, new writes are paused, audio frames already in the input cache are prioritized for emptying, and audio frame output is protected to reduce the impact of continuous backlog. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0019] Figure 1 This is a schematic diagram illustrating the steps of the audio signal data collaborative processing method of the present invention; Figure 2 This is a schematic diagram of a portion of steps S1 in the audio signal data collaborative processing method of the present invention; Figure 3This is a schematic diagram of a portion of steps S2 in the audio signal data collaborative processing method of the present invention; Figure 4 This is a schematic diagram of a portion of steps S3 in the audio signal data collaborative processing method of the present invention; Figure 5 This is a schematic diagram of a portion of step S4 in the audio signal data collaborative processing method of the present invention; Figure 6 This is a schematic diagram of the overall architecture and data flow of the audio signal data collaborative processing system of the present invention; Figure 7 This is a schematic diagram of the timing interaction of the audio signal data collaborative processing system of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0021] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0022] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, the terms first, second, etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0023] This invention provides a method for collaborative processing of audio signal data, such as... Figure 1 As shown, in one embodiment, the method includes: S1. Obtain the periodic coordination basic data of the same audio signal processing link, as well as the audio link load level, delay status flag, output buffer level and resource level execution result of the previous processing cycle, and generate the processing cycle coordination status code of the current processing cycle. S2. Based on the number of valid channels occupied, the number of activated audio processing units, the number of playable samples in the output buffer, the number of times the buffer coverage is insufficient and the number of times the input buffer goes out of bounds, obtain the audio link load adjustment amount, and determine the processing end resource execution level by combining the processing cycle coordination status code. S3. Determine the periodic input buffer sample budget based on the audio link load adjustment, the processing end resource execution level, and the output buffer level. Convert the periodic input buffer sample budget into buffer coverage duration based on the basic sampling rate. Obtain the insufficient buffer coverage duration based on the buffer coverage duration and the actual processing latency. S4. Perform normal processing, mild cache coverage processing, or severe cache coverage processing based on the duration of insufficient cache coverage, and resume processing cycle scheduling after the recovery conditions are met.

[0024] In this embodiment, audio samples are discrete audio data acquired at a base sampling rate. An audio frame is a set of audio samples that enter the same processing task within a processing cycle. For single-channel audio, an audio frame consists of audio samples from that channel entering the input buffer within the current processing cycle; for multi-channel audio, an audio frame consists of audio samples from each valid channel entering the input buffer within the same time range within the current processing cycle. Actual processing latency is timed based on the audio frame corresponding to the current processing cycle.

[0025] In this embodiment, it should be noted that in S1, the processing cycle coordination status code is used to ensure that the acquisition end and the processing end use consistent status criteria in the current processing cycle. The processing cycle coordination status code is a deterministic summary of the operating status of a processing cycle on the same audio signal processing link, and is not used as a random number or communication security parameter. For example, when the base sampling rate is 4.8 × 10⁻⁶... 4 When the parameters are set at samples / s, basic quantization precision of 16 bits, processing cycle length of 20 ms, baseline cycle input buffer sample budget of 960 samples, and maximum cycle input buffer sample budget of 3840 samples, both the acquisition and processing ends read the audio link load level, latency status flag, output buffer level, and resource level execution result of the previous processing cycle, and form status input data according to the same field order. If the previous processing cycle was at the second load level, the latency status flag was normal, the output buffer level was the second level, and the resource level execution result was complete, then the status input data of the current processing cycle can be verified by both ends according to the same rule and mapped to the same processing cycle coordination status code. In this step, the current processing cycle can determine the cycle-level coordination basis based on the cycle status record of the previous processing cycle, reducing the need to re-determine the status of the previous cycle and transmit control data between the acquisition and processing ends.

[0026] In S2, the audio link load adjustment is used to reflect the resource requirements and cache risks of the current processing cycle. This invention uses the number of effective channels occupied, the number of activated audio processing units, the number of playable samples in the output buffer, the number of times the buffer coverage is insufficient, and the number of times the input buffer exceeds its limit in load judgment. For example, in the current processing cycle, the number of effective channels occupied is 4, the maximum number of effective channels is 8, the number of activated audio processing units is 3, the maximum number of audio processing units is 6, the number of playable samples in the output buffer is 1800, the target playback level is 2400 samples, and in the last 10 confirmed processing cycles, the buffer coverage insufficient duration increased once, and the input buffer exceeded its limit once. The above data indicates that the current processing cycle is in a state of medium channel occupancy and medium audio processing unit occupancy, and the output buffer level is lower than the target playback level, with historical records of insufficient buffer coverage increases and input buffer exceeding its limit. Based on this, the processing end obtains the audio link load adjustment and, combined with the processing cycle collaborative status code, determines the processing end resource execution level, so that the processing end resource configuration simultaneously refers to the current load source and the state of the previous processing cycle.

[0027] In S3, the processing end converts the periodic input buffer sample budget into buffer coverage time for comparison based on the same unit of measurement. The periodic input buffer sample budget is the number of samples, e.g., 2400 samples, while the actual processing latency is the duration, e.g., 46 ms or 62 ms. Both need to be converted to the same unit of measurement before comparison. Therefore, the processing end first converts the periodic input buffer sample budget into buffer coverage time based on the base sampling rate. If the base sampling rate is 4.8 × 10⁻⁶... 4 With a periodic input cache sample budget of 2400 samples / s, the cache coverage time is 50 ms. If the actual processing latency recorded by the processor using the same clock source is 46 ms, then the input cache sample budget can cover the processor's processing time; if the actual processing latency is 62 ms, then the processing time exceeds the cache coverage capacity by 12 ms. Through this step, the system unifies the number of input cache samples and the processor's processing time into a time dimension, and determines whether there is insufficient cache coverage in the current processing cycle.

[0028] In S4, tiered link protection is used to prevent minor and major anomalies from being handled using the same coarse-grained approach. If the buffer coverage insufficiency duration does not exceed the first insufficiency boundary, the acquisition end, processing end, and playback output end continue to operate according to the normal processing path. If the buffer coverage insufficiency duration exceeds the first insufficiency boundary but does not exceed the second insufficiency boundary, it indicates that there is a short-term processing backlog at the processing end, but it has not yet reached the point where the entire input buffer write link needs to be paused. At this time, the acquisition end stops writing new audio samples to the input buffer area of ​​the current processing cycle and transfers the subsequently acquired audio samples to the input buffer area or temporary holding area corresponding to the next processing cycle. The processing end transfers the audio frames that have not been processed to the waiting queue of the next processing cycle, and the playback output end outputs short-term holding frames or spare audio frames. If the buffer coverage insufficiency duration exceeds the second insufficiency boundary, it is determined that the current processing cycle has entered the severe buffer coverage insufficiency processing stage. At this time, the acquisition end stops writing new audio samples to the input buffer, the processing end prioritizes emptying the audio frames that have already entered the input buffer, and the playback output end outputs protection audio frames. In this step, the system retains the transfer processing of the acquisition link when the cache coverage is slightly insufficient, and suspends the writing of new input cache when the cache coverage is severely insufficient, thereby reducing the emptying pressure on the processing end.

[0029] The first insufficiency boundary can be determined based on the high quantile of the insufficient buffer coverage time during normal operation cycles, while the second insufficiency boundary can be determined based on the smaller of the protected output allowance time and the time required to empty the input buffer. For example, if the 95th quantile of the insufficient buffer coverage time in 500 consecutive stable processing cycles is 0.006 s, the first insufficiency boundary is taken as 0.008 s after floating up; if the acceptable protected output time at the playback output end is 0.02 s, then the second insufficiency boundary is taken as 0.02 s.

[0030] It should also be noted that the above processing is based on buffer counting, status recording, time recording, and boundary judgment. Input data includes basic periodic coordination data, periodic status data from the previous processing cycle, audio link load status data at the start of the current processing cycle, input completion time, and output completion time. Output data includes processing cycle coordination status codes, audio link load adjustment amounts, processing end resource execution levels, periodic input buffer sample budget, buffer coverage duration, actual processing latency, insufficient buffer coverage duration, and latency status flags. Both the input and output data can be obtained from the buffer counter, periodic status records, and processing end clock records.

[0031] In summary, the entire audio signal data collaborative processing method firstly inherits the audio link load level, latency status flag, output buffer level, and resource allocation execution results from the previous processing cycle through collaborative status codes. This allows the resource configuration and buffer budget of the current processing cycle to reference the confirmed link operation status of the previous processing cycle, rather than adjusting solely based on the current instantaneous data. Furthermore, the audio link load adjustment amount is obtained based on the number of effective channel occupancy, the number of activated audio effect processing units, the number of playable samples in the output buffer, the number of buffer coverage insufficiency increases, and the number of input buffer out-of-bounds increases. This ensures that the load adjustment amount includes the main load sources of the current processing cycle, the output buffer insufficiency situation, and the mismatch trend within the recent confirmed processing cycles. Finally, the input buffer sample budget is converted into buffer coverage time according to the base sampling rate and then compared with the actual processing latency at the processing end using the same dimensions. This allows for determination of whether the input buffer sample budget covers the actual processing time at the processing end and reduces the ambiguity of control boundaries caused by directly comparing sample quantity and time length. Furthermore, based on the duration of insufficient cache coverage, normal processing, mild insufficient cache coverage processing, or severe insufficient cache coverage processing are performed. When there is mild insufficient cache coverage, input cache write transfer, pending queue transfer, and short-term hold frame output are performed. When there is severe insufficient cache coverage, new writes are paused, audio frames already in the input cache are prioritized for emptying, and audio frame output is protected to reduce the impact of continuous backlog.

[0032] like Figure 2 As shown, in one specific embodiment, S1 includes: S11. Obtain the periodic collaborative basic data of the same audio signal processing link; S12. Obtain the cycle status data that has been confirmed in the previous processing cycle; S13. Generate the processing cycle coordination status code for the current processing cycle based on the cycle coordination basic data and the cycle status data confirmed in the previous processing cycle.

[0033] In this embodiment, it should be noted that in S11, the periodic coordination basic data is used to define the operational boundaries of the same audio signal processing link. The periodic coordination basic data includes the basic sampling rate, basic quantization accuracy, processing cycle duration, total input buffer capacity, total output buffer capacity, baseline cycle input buffer sample budget, maximum cycle input buffer sample budget, minimum playback level, target playback level, input buffer occupancy boundary, first insufficient boundary, second insufficient boundary, computational occupancy rate boundary, input buffer occupancy rate boundary, number of recovery confirmation cycles, set of available resource levels at the processing end, processing end resource level boundary data, resource level configuration table, and sample increment table. Among them, the input cache occupancy boundary is used to determine whether an input cache out-of-bounds event has occurred within the most recent confirmed processing cycles; the first insufficiency boundary and the second insufficiency boundary are used to distinguish between normal state, mild cache insufficiency state, and severe cache insufficiency state; the computational occupancy rate boundary, the input cache occupancy rate boundary, and the number of recovery confirmation cycles are used to determine the recovery conditions after severe cache insufficiency processing; the resource level configuration table is used to record the processing thread budget, audio effect processing unit execution budget, data transfer priority, and upper limit of working frequency corresponding to different processing end resource execution levels; the sample increment table is used to record the periodic input cache sample increment corresponding to different audio link load adjustment ranges.

[0034] Furthermore, the base sampling rate represents the number of audio samples per second, the base quantization precision represents the number of bits per audio sample, and the processing cycle duration represents the duration of one coordinated scheduling operation. The total input buffer capacity limits the maximum number of audio samples the processing end's input buffer can hold, and the total output buffer capacity limits the maximum number of processed audio samples the playback output buffer can hold. The baseline cycle input buffer sample budget represents the minimum number of input buffer samples that need to be retained or written in a single processing cycle under low load conditions, while the maximum cycle input buffer sample budget is used to prevent a single processing cycle from consuming excessive input buffer resources. For specific data, the base sampling rate is 4.8 × 10⁻⁶. 4 The parameters are: samples / s, basic quantization precision of 16 bits, processing cycle time of 20 ms, total input buffer capacity and total output buffer capacity of 8192 samples, baseline input buffer sample budget of 960 samples, maximum input buffer sample budget of 3840 samples, minimum playback level of 1200 samples, and target playback level of 2400 samples. These data can be obtained from the audio signal processing link configuration and buffer capacity and used for subsequent calculations.

[0035] The minimum and target playback levels are determined based on the safe time allowed for continuous reading at the playback output, which is provided by the device initialization test or factory configuration. For example, the base sampling rate is 4.8 × 10⁻⁶. 4For samples / s, if the minimum safe read time is 25 ms, the minimum playback threshold is 1200 samples; if the target safe read time is 50 ms, the target playback threshold is 2400 samples. The input buffer occupancy boundary is determined based on the total input buffer capacity and reserved empty space. Reserved empty space is used to allow buffer space for incomplete audio frame transfers. For example, if the total input buffer capacity is 8192 samples, and the maximum allowed occupancy rate is 80%, the input buffer occupancy boundary is 6553 samples. Exceeding this boundary is recorded as an input buffer outage. The computational occupancy boundary is determined based on processor occupancy records during stable operation. Processing cycles without insufficient buffer coverage are selected as samples, and a fixed margin is added to the high quantile occupancy rate. For example, if the 95th percentile of processor occupancy in 500 consecutive stable processing cycles is 75%, after a 5% increase, the computational occupancy boundary is set to 80%. The input cache occupancy rate boundary is determined based on the ratio of the number of input cache entries to the total input cache capacity, and is used for recovery condition judgment. For example, if the total input cache capacity is 8192 samples, and the recovery phase requires the number of input cache entries to be less than 6144 samples, then the input cache occupancy rate boundary is set to 75%. When it is below this boundary, the input cache is considered to have met the conditions for recovery writing. The number of recovery confirmation cycles is determined based on the processing cycle duration and the recovery observation time. The recovery observation time is used to exclude short-term fluctuations in a single cycle. For example, if the processing cycle duration is 20 ms, and the recovery observation time is 60 ms, then the number of recovery confirmation cycles is 3. After three consecutive detection cycles meet the recovery conditions, the processing cycle scheduling is resumed. The set of available resource levels on the processing end is determined based on the number of available threads on the processor, the number of concurrent audio processing units, the number of data transfer queues, and the operating frequency level, and is written into the cycle coordination basic data during the initialization phase. For example, when the processing end supports 1 to 3 processing threads, a maximum of 6 audio processing units, and 3 operating frequency levels, it is divided into the first resource execution level, the second resource execution level, and the third resource execution level. The resource tier configuration table is pre-written based on the processing hardware capabilities and the concurrent limit of audio processing tasks. For example, the first resource tier configuration is 1 processing thread, 3 audio processing units, normal transfer priority, and a working frequency limit of 0.8 GHz; the second resource tier configuration is 2 processing threads, 4 audio processing units, higher transfer priority, and a working frequency limit of 1.1 GHz; and the third resource tier configuration is 3 processing threads, 6 audio processing units, highest transfer priority, and a working frequency limit of 1.4 GHz.

[0036] In S12, the cycle status data confirmed in the previous processing cycle is used to reflect the actual operating results of the most recent processing cycle. The cycle status data includes the audio link load level, latency status flag, output buffer level, and resource allocation execution result of the previous processing cycle. The audio link load level includes a first load level, a second load level, and a third load level, defined based on whether the audio link load adjustment falls within the range defined by the processing end resource allocation boundary data. The latency status flag includes a normal state, a slightly insufficient buffer coverage state, and a severely insufficient buffer coverage state. A normal state indicates that the insufficient buffer coverage duration does not exceed the first insufficient boundary; a slightly insufficient buffer coverage state indicates that the insufficient buffer coverage duration exceeds the first insufficient boundary but does not exceed the second insufficient boundary; and a severely insufficient buffer coverage state indicates that the insufficient buffer coverage duration exceeds the second insufficient boundary. The output buffer level includes a first level, a second level, and a third level, corresponding to the number of playable samples in the output buffer being below the minimum playback level, not below the minimum playback level but not reaching the target playback level, and reaching the target playback level, respectively. The resource allocation execution results include an execution completion flag, actual operating frequency level, processing thread occupancy level, and sound effect processing unit activation level. All of the above periodic status data comes from completed execution records of the previous processing cycle and does not depend on data that is not yet completed in the current processing cycle.

[0037] Specifically, for the first processing cycle after device startup, if there is no confirmed cycle status data from the previous processing cycle, the status of the previous processing cycle is initialized to the first load level, normal state, second water level level, and completed state, and this initialization status is written to the cycle status record. If the number of playable samples in the output buffer is lower than the minimum playback water level at startup, the output buffer water level is initialized to the first water level level; if the number of playable samples in the output buffer is not lower than the minimum playback water level but has not reached the target playback water level, the output buffer water level is initialized to the second water level level; if the number of playable samples in the output buffer reaches the target playback water level at startup, the output buffer water level is initialized to the third water level level. Through the above initialization rules, the first processing cycle can enter the scheduling process according to the same status code generation logic as subsequent processing cycles.

[0038] In S13, the acquisition and processing ends read the same initial link configuration data, current processing cycle number, audio link load level of the previous processing cycle, delay status flag of the previous processing cycle, output buffer level of the previous processing cycle, and resource level execution result of the previous processing cycle, and concatenate them into status input data according to the same field order. Subsequently, the acquisition and processing ends encode the audio link load level of the previous processing cycle, delay status flag of the previous processing cycle, output buffer level of the previous processing cycle, and resource level execution result of the previous processing cycle in the status input data to obtain status fields; perform deterministic verification operations on the status input data to obtain verification fields; combine the status fields and verification fields according to a preset field order and map them to the range of values ​​for the processing cycle collaborative status code to obtain the processing cycle collaborative status code for the current processing cycle. The deterministic verification operation can employ cyclic redundancy check (CRC), checksum operation, or fixed digest operation, ensuring that the same input corresponds to the same output. For example, if the current processing cycle number is 128, the audio link load level of the previous processing cycle is the second load level, the latency status flag is normal, the output buffer level is the second level, and the resource allocation result is complete, then the acquisition end and the processing end can obtain the same status input data based on the same field sequence and generate a consistent processing cycle collaborative status code. This step breaks down the status code generation process into three steps: input data concatenation, deterministic verification, and value range mapping, making the status code generation process have clear inputs and outputs.

[0039] The initial link configuration data consists of the link identifier, base sampling rate, base quantization precision, maximum number of effective channels, maximum number of audio processing units, total input buffer capacity, total output buffer capacity, and device configuration version number, and remains unchanged after being read during the device initialization phase; for example, if the base sampling rate is 4.8 × 10⁻⁶... 4 If the maximum number of valid channels is 8 and the total input buffer capacity is 8192 samples / s, then the above data is written into the initial link configuration data. Preset field order is written to the configuration area during the initialization phase. The field order is as follows: initial link configuration data, current processing cycle number, audio link load level, latency status flag, output buffer level, and resource level execution result. The range of processing cycle coordination status codes is determined by the width of the status code field; for example, when the status code field width is 16 bits, the range of processing cycle coordination status codes is 0 to 65535. The current processing cycle number is generated sequentially by the processing end at the start of scheduling and corresponds one-to-one with the cycle status record; for example, the first processing cycle number after device startup is 1, and the number increments by 1 after each 20 ms processing cycle; if the previous processing cycle number was 127, then the current processing cycle number is 128.

[0040] In a supplementary implementation, if the acquisition end and the processing end are located within the same processing chip, the processing cycle coordination status code can be directly generated from the cycle status records in the same storage area. If the acquisition end and the processing end are located within different processing units, they can save the same initial link configuration data during the initialization phase and synchronize the confirmed cycle status data at the end of each processing cycle. Since the processing cycle coordination status code uses data confirmed in the previous processing cycle, the impact of the current processing cycle's acquisition or processing status on the consistency of the results generated by both ends can be reduced. If inconsistencies are detected between the processing cycle coordination status codes generated by the acquisition end and the processing end, the delay status flag of the current processing cycle can be recorded as a mild cache under-coverage state, and the processing cycle coordination status code can be regenerated after rereading the cycle status record of the previous processing cycle, so that subsequent processing cycles use a consistent coordination basis.

[0041] like Figure 3 As shown, in one specific embodiment, S2 includes: S21. Obtain the audio link load status data for the current processing cycle; S22. Obtain the audio link load adjustment amount for the current processing cycle based on the audio link load status data; S23. Determine the execution level of the processing end resources for the current processing cycle based on the audio link load adjustment amount and the processing cycle coordination status code.

[0042] In this embodiment, it should be noted that in S21, the audio link load status data includes the number of effective channel occupancy in the current processing cycle, the number of enabled audio processing units, the number of playable samples in the output buffer, the number of times buffer coverage is insufficient, and the number of times the input buffer exceeds the limit. The number of effective channel occupancy represents the number of audio channels actually participating in processing in the current processing cycle; the number of enabled audio processing units represents the number of audio processing units participating in digital signal processing in the current processing cycle; and the number of playable samples in the output buffer represents the number of audio samples that have been processed and are available for reading in the output buffer of the playback output terminal. The number of times buffer coverage is insufficient represents the number of times the insufficient buffer duration has increased relative to the previous processing cycle in the most recent confirmed processing cycles; the number of times the input buffer exceeds the input buffer occupancy boundary in the most recent confirmed processing cycles. For example, if the insufficient buffer duration increases from 0 ms to 4 ms once in the most recent 10 confirmed processing cycles, and the input buffer occupancy exceeds 80% of the total input buffer capacity once, then the number of times buffer coverage is insufficient is 1, and the number of times the input buffer exceeds the limit is 1. These data can be obtained from the buffer counter, periodic status records, and audio processing unit activation records at the processing end.

[0043] The audio link load status data is read once at the start of the current processing cycle and serves as the fixed input data for obtaining the audio link load adjustment within the current processing cycle. Changes in channel activation, audio effect processing unit activation, and the number of playable samples in the output buffer that occur after the start of the current processing cycle are not written back to the audio link load adjustment in the current processing cycle, but are statistically analyzed in the audio link load status data of the next processing cycle. By using a unified reading time point, the acquisition end, processing end, and playback output end use the load status data corresponding to the same reading time point within the same processing cycle.

[0044] The number of valid channels occupied is determined based on the channel enable flag and the audio samples entering the input buffer during the current processing cycle. For any audio channel, if the channel enable flag is enabled and there are audio samples entering the input buffer for that audio channel during the current processing cycle, then that audio channel is counted in the number of valid channels occupied; if the channel enable flag is disabled, or there are no audio samples entering the input buffer for that audio channel during the current processing cycle, then it is not counted in the number of valid channels occupied. The number of activated sound effect processing units is determined based on the sound effect processing unit enable flag and the processing task table for the current processing cycle; if a sound effect processing unit is called by the processing task table during the current processing cycle, then it is counted in the number of activated sound effect processing units; otherwise, it is not counted in the number of activated sound effect processing units.

[0045] Specifically, when the number of confirmed historical processing cycles is less than the number W of historical processing cycles used for trend statistics, the processing end only performs statistics based on the confirmed historical processing cycles. If no confirmed historical processing cycle exists, the number of times the buffer coverage is insufficient and the number of times the input buffer exceeds the limit are both set to 0. If only one confirmed historical processing cycle exists, there is no comparable previous processing cycle for the duration of buffer coverage insufficiency, the number of times the buffer coverage is insufficient is set to 0, and the number of times the input buffer exceeds the limit is counted based on whether the input buffer usage exceeds the input buffer usage boundary in the confirmed historical processing cycle. Through the above rules, the audio link load adjustment can also be determined during the initial stage of device startup.

[0046] Specifically, when counting the number of times the cache coverage is insufficient, the processing end reads the most recent confirmed processing cycle record in ascending order of processing cycle number, and compares the cache coverage insufficient duration of each confirmed processing cycle with the cache coverage insufficient duration of the adjacent previous confirmed processing cycle. If the cache coverage insufficient duration of the current confirmed processing cycle is greater than the cache coverage insufficient duration of the adjacent previous confirmed processing cycle, it is counted as one increase; if the cache coverage insufficient duration of the current confirmed processing cycle is less than or equal to the cache coverage insufficient duration of the adjacent previous confirmed processing cycle, it is not counted as an increase.

[0047] The maximum number of effective channels is determined based on the channel configuration table of the audio link, and the maximum number of audio effect processing units is determined based on the total number of audio effect processing units that the processing end is allowed to call concurrently. For example, if there are 8 available input channels in the channel configuration table, the maximum number of effective channels is 8; if the processing end allows a maximum of 6 audio effect processing units to be called simultaneously, the maximum number of audio effect processing units is 6. The number of historical processing cycles W used for trend statistics is determined based on the trend observation time and the processing cycle duration. The trend observation time is used to cover short-term load fluctuations. For example, if the processing cycle duration is 20 ms, and the trend observation time is 200 ms, then W is 10, which means counting the number of times the buffer coverage is insufficient and the number of times the input buffer goes out of bounds in the most recent 10 confirmed processing cycles.

[0048] The channel activation flag is determined by the audio input routing configuration and the current acquisition channel status, while the audio effect processing unit activation flag is determined by the audio effect functions enabled by the user and the current processing task list. For example, if 4 out of 8 input channels are enabled and all have samples entering the input buffer, then the number of effective channels occupied is 4; if the processing task list calls speech enhancement, echo suppression, and equalization processing, then the number of activated audio effect processing units is 3.

[0049] In S22, the processing unit obtains the audio link load adjustment amount for the current processing cycle based on the audio link load status data. The audio link load adjustment amount characterizes the overall demand of the same audio signal processing link on the processing unit's resources and input buffer budget in the current processing cycle. The audio link load adjustment amount for the current processing cycle is determined according to the following expression:

[0050] in, Indicates the first Audio link load adjustment per processing cycle; Indicates the current processing cycle number; This indicates a minimum value operation, used to limit the audio link load adjustment to no more than [a certain value]. Within the range; This indicates the upper limit of the audio link load adjustment; This indicates that the maximum value among the items within the square brackets is taken to determine the link load item with the largest value in the current processing cycle. Indicates the first The number of valid channels occupied within a processing cycle, in terms of channel count; This indicates the maximum number of valid channels allowed in the same audio signal processing link, expressed in the number of channels. Indicates the channel occupancy rate; Indicates the first The number of audio processing units activated within a processing cycle, in units of processing units; This indicates the maximum number of audio processing units that can be used simultaneously on the processing end, expressed in the number of processing units. Indicates the proportion of audio processing unit occupied; Indicates the first The number of samples that can be played in the output buffer corresponding to each processing cycle, in units of sample count; This indicates the number of samples that can be played in the output buffer corresponding to the target playback level, in units of sample count. This indicates the output buffer level ratio; 0 indicates the lower limit of insufficient output buffer level. Indicates the degree of insufficiency in the output buffer; Indicates as of the date The most recent processing cycle before Within a confirmed processing cycle, the number of times the cache coverage insufficiency duration increased relative to the previous processing cycle, in terms of the number of times; This is a statistic indicating an increasing duration of insufficient cache coverage. This indicates the number of historical processing cycles used for trend statistics, with the unit being the number of processing cycles. This indicates an increase in the percentage of insufficient cache coverage. Indicates as of the date Within the W most recent confirmed processing cycles prior to the current processing cycle, the number of times the input buffer usage exceeded the input buffer usage boundary, in terms of the number of times; This indicates the percentage of input buffer overflows.

[0051] In the above expressions, the channel occupancy ratio, audio effect processing unit occupancy ratio, and output buffer insufficiency level correspond to the input-side parallel scale, processing-side functional complexity, and output-side continuity risk, respectively. First, the maximum value of these three is taken to determine the main load source of the current processing cycle and reduce the impact of multiple similar load items overlapping on resource adjustment. Then, the buffer coverage insufficiency increase ratio and input buffer out-of-bounds ratio are added to ensure that the audio link load adjustment not only reflects the current processing cycle status but also the mismatch trend in the most recent confirmed processing cycles. For example, if... , , , , , , , , The channel occupancy rate is 0.5, the audio processing unit occupancy rate is 0.5, the output buffer level insufficiency is 0.25, the main load value is 0.5, the buffer coverage insufficiency increase rate is 0.1, the input buffer out-of-bounds rate is 0.1, and the audio link load adjustment is 0.7. This calculation result indicates that the current processing cycle has not reached the maximum load and corresponds to the medium link load range applicable to the second resource execution level.

[0052] In S23, the processing end determines the processing end resource execution level for the current processing cycle based on the audio link load adjustment and the processing cycle coordination status code. The processing end resource execution levels include a first resource execution level, a second resource execution level, and a third resource execution level. The first, second, and third resource execution levels correspond to different processing thread budgets, audio effect processing unit execution budgets, data transport priorities, and upper limits of operating frequencies, respectively. The processing end reads the delay status flag from the previous processing cycle based on the processing cycle coordination status code and prioritizes the determination according to the order of third resource execution level, second resource execution level, and first resource execution level. When the audio link load adjustment is within the third range, or when a severe cache coverage shortage occurred in the previous processing cycle and the recovery conditions have been met, the system is designated as the third resource execution level. When the conditions for the third resource execution level are not met, and the audio link load adjustment is within the second range, or when a mild cache coverage shortage occurred in the previous processing cycle, the system is designated as the second resource execution level. When the conditions for both the third and second resource execution levels are not met, and the audio link load adjustment is within the first range and no cache coverage shortage occurred in the previous processing cycle, the system is designated as the first resource execution level. If a severe cache coverage shortage occurred in the previous processing cycle and the recovery conditions were not met, the current processing cycle will continue to process the severe cache coverage shortage and will not resume normal processing cycle scheduling. If the processing end resource level boundary data corresponds to the first range (0 to 0.45), the second range (greater than 0.45 but not greater than 0.75), and the third range (greater than 0.75 but not greater than 1), then the aforementioned audio link load adjustment of 0.7 corresponds to the second resource execution level. Through this logic, the execution level of the processing end resources simultaneously refers to the current audio link load adjustment and the status of the previous processing cycle carried by the processing cycle coordination status code.

[0053] The processing end resource tier boundary data is determined based on the results of a limited number of load tests. During the tests, the audio link load adjustment, actual processing latency, and whether insufficient buffer coverage occurred are recorded. For example, if... If no cache overwrite occurs for 300 consecutive processing cycles when the cache value does not exceed 0.45, then 0 to 0.45 is set as the first range; if... If the latency starts to rise after it exceeds 0.45, but can still be mitigated by increasing resource coverage, then the range of latency greater than 0.45 but not greater than 0.75 is set as the second range; higher loads are set as the third range.

[0054] The sample increment table is determined based on actual processing latency test results under different load ranges, ensuring that the increased periodic input buffer sample budget can cover the processing time under the corresponding load. For example, the baseline periodic input buffer sample budget is 960 samples. When falling into the second range, the sample increment is 1440 samples, and the periodic input buffer sample budget is 2400 samples; when When falling into the third range, the sample increment can be 2880 samples, and is limited by the maximum periodic input cache sample budget.

[0055] Specifically, when the output buffer level is at the third level, it indicates that the playback output end has sufficient buffer capacity, and the increase in the periodic input buffer sample budget follows the lower-level increment in the sample increment table. For example, if If the sample falls into the second range and the corresponding increment in the sample increment table is 1440 samples, but the output buffer level is the third level, then only 960 samples will be added, and the periodic input buffer sample budget will be determined to be 1920 samples.

[0056] The execution result of a resource slot is generated by the resource scheduling record at the end of the current processing cycle. The record includes whether it is completed, the actual working frequency level, the processing thread occupancy level, and the audio effect processing unit activation level. For example, if the second resource execution slot requires 2 processing threads and 4 audio effect processing units, and the current processing cycle actually activates 2 threads and 4 audio effect processing units and completes the audio frame processing, then it is recorded as completed.

[0057] In a supplementary implementation, the execution result of the resource execution level is written to the cycle status record after the current processing cycle ends. If the current processing cycle is determined to be the second resource execution level, the processing end can configure the second processing thread budget, the second audio effect processing unit execution budget, the second data transfer priority, and the second working frequency upper limit. If the processing end fails to complete the processing of the corresponding audio frame within the processing cycle duration, the resource execution result is recorded as incomplete; if the processing end completes the corresponding resource configuration and completes the audio processing task of the current processing cycle, the resource execution result is recorded as complete. The resource execution result enters the processing cycle collaborative status code generation process in the next processing cycle, enabling the next processing cycle to reflect the actual execution status of the resource level.

[0058] like Figure 4As shown, in one specific embodiment, S3 includes: S31. Determine the periodic input buffer sample budget for the current processing cycle based on the audio link load adjustment, the processing end resource execution level, and the output buffer level. S32. Obtain the cache coverage duration based on the periodic input cache sample budget and the basic sampling rate, and obtain the insufficient cache coverage duration based on the cache coverage duration and the actual processing latency. S33. Save the cycle status record of the current processing cycle.

[0059] In this embodiment, it should be noted that in S31, the periodic input cache sample budget represents the number of input cache samples allowed to enter the expected processing range of the current processing cycle, including newly written audio samples in the current processing cycle and unprocessed audio samples transferred from the previous processing cycle. The sum of the number of newly written audio samples in the current processing cycle and the number of unprocessed audio samples transferred from the previous processing cycle does not exceed the periodic input cache sample budget of the current processing cycle. The periodic input cache sample budget must not be lower than the baseline periodic input cache sample budget and must not be higher than the maximum periodic input cache sample budget. When the processing end resource execution level is the first resource execution level and the output cache level is the second or third level, the periodic input cache sample budget of the current processing cycle is determined based on the baseline periodic input cache sample budget. When the processing end resource execution level is the second resource execution level, the periodic input cache sample budget of the current processing cycle is determined based on the baseline periodic input cache sample budget and the sample increment corresponding to the audio link load adjustment amount. When the processing end resource execution level is the third resource execution level, or the output buffer level is the first level, the periodic input buffer sample budget for the current processing cycle is increased. When the output buffer level is the third level, the increase in the periodic input buffer sample budget for the current processing cycle is limited. For example, if the baseline periodic input buffer sample budget is 960 samples, the maximum periodic input buffer sample budget is 3840 samples, the current processing end resource execution level is the second resource execution level, and the output buffer level is the second level, then the processing end can determine the periodic input buffer sample budget to be 2400 samples based on the sample increment corresponding to the audio link load adjustment. This budget is greater than the baseline periodic input buffer sample budget and does not exceed the maximum periodic input buffer sample budget, providing buffer coverage time for moderate link loads.

[0060] In S32, the processing end first converts the periodic input buffer sample budget into the buffer coverage duration for the current processing cycle based on the periodic input buffer sample budget and the base sampling rate. Then, it obtains the buffer coverage insufficiency duration based on the actual processing latency and the buffer coverage duration. The processing end records the input completion time when the audio frame corresponding to the current processing cycle is written to the input buffer, and records the output completion time when the audio frame completes digital signal processing and is written to the output buffer. The time interval between the output completion time and the input completion time is the actual processing latency. The buffer coverage insufficiency duration for the current processing cycle is determined according to the following expression:

[0061] in, Indicates the first The duration of insufficient cache coverage for a single processing cycle, in seconds; Indicates the current processing cycle number; This indicates the maximum value operation, which is used to determine the insufficient cache coverage time as zero when the actual processing latency does not exceed the cache coverage time. This indicates the lower limit of the duration for which the cache is not fully covered. Indicates the first The actual processing delay per processing cycle, in seconds; Indicates the first The cycle input buffer sample budget for each processing cycle, in units of sample count; This represents the base sampling rate, measured in samples per second. Indicates the first Cache coverage duration per processing cycle, in seconds; index This indicates the basic link configuration.

[0062] The above expression converts the sample size into a time duration before comparing it with the actual processing delay, thus addressing the issue of data from different units not being directly subtractable. If , The cache coverage time is .like If the actual processing latency does not exceed the cache overwrite time, .like If the actual processing latency exceeds the cache overwrite time, then the actual processing latency will exceed the cache overwrite time. , .like If the actual processing latency exceeds the cache overwrite time, then the actual processing latency will exceed the cache overwrite time. , This calculation process can directly output the length of time during which the input buffer coverage is insufficient in the current processing cycle, providing a clear basis for judgment in the hierarchical link protection of S4.

[0063] In S33, the processing end stores the cycle status record for the current processing cycle. The cycle status record includes the processing cycle number, processing cycle coordination status code, audio link load adjustment, processing end resource execution level, cycle input buffer sample budget, buffer coverage duration, actual processing latency, buffer under-coverage duration, output buffer level, and resource level execution result. The cycle status record is used for generating the processing cycle coordination status code for the next processing cycle, counting the number of buffer under-coverage increases, counting the number of input buffer out-of-bounds increases, and determining recovery conditions. For example, if the current processing cycle number is 128, the audio link load adjustment is 0.7, the processing end resource execution level is the second resource execution level, the cycle input buffer sample budget is 2400 samples, the buffer coverage duration is 0.05 s, the actual processing latency is 0.062 s, the buffer under-coverage duration is 0.012 s, and the output buffer level is the second level, then all of the above data is written to the cycle status record. When counting the number of times the cache coverage is insufficient in the next processing cycle, the duration of the current processing cycle's insufficient cache coverage can be compared with the duration of the previous processing cycle's insufficient cache coverage. If there is an increase, it will be counted as an increase in the number of times the cache coverage is insufficient.

[0064] The maximum capacity of the periodic status record is determined based on the number of historical processing cycles (W) used for trend statistics and the number of recovery confirmation cycles, and a record number not less than the larger of the two values ​​is retained. For example, if W=10 and the number of recovery confirmation cycles is 3, the capacity of the circular record table can be set to 16 records, which can cover the data required for trend statistics and recovery judgment.

[0065] In a supplementary implementation, the cycle status record can use a circular record table to store data from the most recent confirmed processing cycles. A record is written at the end of each processing cycle, and the oldest record is overwritten when the record table reaches its capacity limit. In this way, the processor can count the number of times the cache coverage becomes insufficient and the number of times the input cache overflows within a fixed storage space, without needing to store long-term historical data or access external training samples. If the current processing cycle fails to complete normal output due to severe cache coverage issues, the cycle status record still stores the latency status flag, resource allocation execution result, and recovery detection result for the current processing cycle, so that deterministic scheduling can continue in the next processing cycle.

[0066] like Figure 5 As shown, in one specific embodiment, S4 includes: S41. When the cache coverage is insufficient for a period of time but does not exceed the first insufficient boundary, normal processing is performed. S42. When the cache coverage insufficiency duration exceeds the first insufficiency boundary but does not exceed the second insufficiency boundary, perform mild cache coverage insufficiency processing. S43. When the cache coverage shortage duration exceeds the second insufficiency boundary, perform severe cache coverage shortage processing and resume processing cycle scheduling after the recovery conditions are met.

[0067] In this embodiment, it should be noted that in S41, "normal processing" means that the periodic input buffer sample budget of the current processing cycle covers the actual processing delay of the processing end, or the buffer coverage insufficiency time does not exceed the first insufficiency boundary. Under normal processing, the acquisition end writes audio samples to the input buffer according to the periodic input buffer sample budget of the current processing cycle; the processing end completes digital signal processing according to the processing end resource execution level of the current processing cycle; the playback output end reads the processed audio samples from the output buffer according to the normal playback rhythm; and the processing end records the delay status flag of the current processing cycle as normal. For example, if the first insufficiency boundary is 0.008 s, and the buffer coverage insufficiency time is 0 s or 0.004 s, then the current processing cycle performs normal processing. The result of normal processing is written to the periodic status record and used as input for generating the processing cycle coordinating status code in the next processing cycle. Since normal processing does not involve the transfer of incomplete audio frames or the short-term holding of frame output, the corresponding periodic scheduling path adds fewer additional processing steps.

[0068] In S42, mild buffer coverage insufficiency processing indicates that the current processing cycle has experienced buffer coverage insufficiency, but the degree of insufficiency has not yet reached the trigger condition for severe buffer coverage insufficiency processing. If the first insufficiency boundary is 0.008 s, the second insufficiency boundary is 0.02 s, and the buffer coverage insufficiency duration of the current processing cycle is 0.012 s, then the current processing cycle enters mild buffer coverage insufficiency processing. Under mild buffer coverage insufficiency processing, the acquisition end stops writing new audio samples to the input buffer area of ​​the current processing cycle and transfers the subsequently acquired audio samples to the input buffer area or temporary holding area corresponding to the next processing cycle; the processing end transfers the audio frames that have not yet been processed in the current processing cycle to the waiting queue of the next processing cycle, and increases the processing end resource execution level or increases the cycle input buffer sample budget in the next processing cycle; the playback output end outputs the short-term holding frame or spare audio frame at the end of the previous processing cycle; the processing end records the delay status flag of the current processing cycle as mild buffer coverage insufficiency state. Short-term hold frames are copied from the last consecutive audio samples in the output buffer of the previous processing cycle. The number of samples in a short-term hold frame is determined by the number of output samples that need to be supplemented in the current processing cycle, and does not exceed the number of samples corresponding to one processing cycle. If the number of copyable last audio samples in the output buffer of the previous processing cycle is insufficient, existing last audio samples are copied first, and then supplemented with zero-amplitude samples. Backup audio frames are obtained by decreasing the amplitude of the last consecutive audio samples in the output buffer of the previous processing cycle. The amplitude decreasing process gradually reduces the sample amplitude according to the sample output order until the amplitude of the last sample in the backup audio frame is reduced to zero or close to zero. If no usable last audio samples exist in the previous processing cycle, the backup audio frame is generated using zero-amplitude samples. This processing method retains audio frames already in the input buffer and maintains link continuity through queue transfer and output protection.

[0069] In this process, unprocessed audio frames carried over to the next processing cycle occupy the cycle input buffer sample budget for that cycle. Before writing new audio samples at the start of the next processing cycle, the processor first counts the number of samples contained in the unprocessed audio frames carried over, and uses the remaining portion after deducting this number from the cycle input buffer sample budget as the number of new samples allowed to be written in the next processing cycle.

[0070] The number of output samples that need to be supplemented in the current processing cycle is determined based on the number of output samples that should be output in the current processing cycle and the number of readable samples in the output buffer; for example, the processing cycle duration is 20 ms and the base sampling rate is 4.8 × 10⁻⁶. 4The current processing cycle should output 960 samples per second. If only 600 samples are available in the output buffer, then 360 samples need to be supplemented. The number of samples for short-term hold frames, spare audio frames, or guard audio frames should not exceed 360 samples. Amplitude reduction processing is performed in equal-step decreasing order according to the output sample order, ensuring that the first spare sample maintains its original amplitude, and the last spare sample is reduced to zero or close to zero. For example, if a spare audio frame contains 480 samples, the first sample is output at 100% of its original amplitude, the intermediate samples decrease sequentially, and the last sample is output at 0% of its original amplitude. The number of zero-amplitude samples is the same as the number of output samples that need to be supplemented in the current processing cycle, and does not exceed the number of samples corresponding to one processing cycle. For example, if the current processing cycle needs to supplement 360 samples, and there are no available last audio samples in the previous processing cycle, then 360 zero-amplitude spare audio frames are generated. If 1200 samples need to be supplemented, then a maximum of 960 samples corresponding to one processing cycle will be generated.

[0071] In S43, severe cache insufficiency processing indicates that the actual processing latency of the current processing cycle has significantly exceeded the cache coverage capacity. If the second insufficiency boundary is 0.02 s, and the cache insufficiency duration of the current processing cycle is 0.026 s, then the current processing cycle enters severe cache insufficiency processing. Under severe cache insufficiency processing, the acquisition end pauses writing new audio samples to the input buffer; the processing end prioritizes emptying audio frames already in the input buffer, pauses receiving new input buffer write requests, and detects the computational utilization rate, input buffer utilization rate, output buffer level, processing thread blocking status, and memory usage status; the playback output end outputs a protection audio frame; the processing end records the latency status flag of the current processing cycle as severe cache insufficiency. Recovery conditions include computational utilization rate below the computational utilization rate boundary, input buffer utilization rate below the input buffer utilization rate boundary, output buffer level at the second or third level, and no processing thread blocking status detected. When the processing end meets the recovery conditions for multiple consecutive detection cycles, the acquisition end resumes writing audio samples to the input buffer, and the processing end re-enters the processing cycle scheduling. This processing method suspends the writing of new input buffers when there is a backlog at the processing end, and reduces the risk of data loss by emptying the audio frames that have already entered the input buffer.

[0072] The guard audio frame is used to maintain safe output from the playback output during periods of severe buffer overload. The guard audio frame is preferentially generated from the last consecutive audio samples in the output buffer of the previous processing cycle through amplitude reduction processing; if no usable last audio samples exist in the output buffer of the previous processing cycle, the guard audio frame is generated from zero-amplitude samples. The number of samples in the guard audio frame does not exceed the number of samples corresponding to one processing cycle, and is determined based on the number of output samples that the playback output needs to supplement in the current processing cycle.

[0073] The processing thread blocking state is determined based on changes in the dequeue length of the pending queue and the thread running flag. For example, if the pending queue length does not decrease for two consecutive detection cycles, and the corresponding processing thread running flag remains busy, it is recorded as a detected processing thread blocking state. If the pending queue length decreases in any detection cycle, it is recorded as no detected processing thread blocking state. Memory usage is determined based on the ratio of currently allocated audio buffer space to available audio memory space, and is used to assist in determining whether to continue pausing new input buffer write requests. For example, if the available audio memory space is 80 MB and the currently allocated audio buffer space is 64 MB, the memory usage rate is 80%. If this value exceeds the preset memory usage boundary, the paused new write state is maintained.

[0074] In a supplementary embodiment, the first and second insufficiency boundaries can be determined based on stable operation data of the same audio signal processing link. During stable operation, the processing end records the buffer coverage insufficiency duration for multiple normal processing cycles and selects a boundary that distinguishes between normal processing and mild buffer coverage insufficiency states as the first insufficiency boundary. The second insufficiency boundary is determined based on the acceptable protection output time of the playback output end and the time required for the processing end to empty the input buffer, and the second insufficiency boundary is greater than the first insufficiency boundary. The first and second insufficiency boundaries can be written into the cycle coordination basic data for the processing end to perform state determination at the end of each processing cycle.

[0075] like Figure 6 and Figure 7As shown, this invention also provides an audio signal data collaborative processing system, including: a periodic status generation module, a resource level determination module, a buffer budget determination module, and a hierarchical link protection module. The periodic status generation module is used to acquire the periodic collaborative basic data of the same audio signal processing link, as well as the audio link load level, delay status flag, output buffer level, and resource level execution result of the previous processing cycle, and generate the processing cycle collaborative status code for the current processing cycle. The resource level determination module is used to obtain the audio link load adjustment amount based on the number of effective channel occupancy, the number of activated audio effect processing units, the number of playable samples in the output buffer, the number of buffer coverage insufficiency rises, and the number of input buffer out-of-bounds occurrences, and determines the processing end resource execution level in conjunction with the processing cycle collaborative status code. The buffer budget determination module is used to determine the periodic input buffer sample budget based on the audio link load adjustment amount, the processing end resource execution level, and the output buffer level, convert the periodic input buffer sample budget into buffer coverage duration according to the basic sampling rate, and obtain the buffer coverage insufficiency duration based on the buffer coverage duration and the actual processing delay. The hierarchical link protection module is used to perform normal processing, mild cache coverage processing, or severe cache coverage processing based on the duration of insufficient cache coverage, and resume the processing cycle scheduling after the recovery conditions are met.

[0076] In one implementation, the cycle status generation module is used to concatenate the initial link configuration data, the current processing cycle number, the audio link load level of the previous processing cycle, the delay status flag of the previous processing cycle, the output buffer level of the previous processing cycle, and the resource level execution result of the previous processing cycle in the same field order, and generate a processing cycle collaborative status code through deterministic verification. The resource level determination module is used to obtain the audio link load adjustment amount based on the channel occupancy ratio, the audio effect processing unit occupancy ratio, the degree of output buffer level insufficiency, the increase ratio of buffer coverage insufficiency, and the input buffer out-of-bounds ratio, and to determine the processing thread budget, audio effect processing unit execution budget, data transfer priority, and upper limit of working frequency based on the audio link load adjustment amount and the processing cycle collaborative status code. The buffer budget determination module is used to correct the cycle input buffer sample budget based on the output buffer level, convert the cycle input buffer sample budget into buffer coverage duration, and compare it with the actual processing delay in the same dimension to obtain the buffer coverage insufficiency duration. The hierarchical link protection module is used to output short-term hold frames or backup audio frames under mild buffer overlay processing, and to suspend writing new audio samples to the input buffer, prioritize emptying audio frames that have already entered the input buffer, and output protection audio frames under severe buffer overlay processing.

[0077] This invention also provides an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, periodic coordination basic data, initial link configuration data, periodic status records, processing end resource level boundary data, a first insufficiency boundary, and a second insufficiency boundary. When the processor executes the computer program, it implements the aforementioned audio signal data coordination processing method. The processor can be a central processing unit, a digital signal processor, a microcontroller, a field-programmable gate array, or other processor used for performing audio data processing and periodic scheduling control. The electronic device can be a portable audio device, a real-time communication device, an audio acquisition device, an audio processing device, or a terminal device with audio acquisition, processing, and playback output functions.

[0078] The present invention also provides a non-transitory computer-readable storage medium. A computer program is stored on the non-transitory computer-readable storage medium. When the computer program is executed by a processor, it implements the above-described audio signal data collaborative processing method. The non-transitory computer-readable storage medium can be a read-only memory, random access memory, flash memory, magnetic disk, optical disk, or other media used for storing computer programs.

[0079] To further clarify the operating mechanism and physical quantification process of the technical solution of this invention, the following analysis will be conducted in detail on the underlying derivation logic of the audio signal data collaborative processing method, system, device and medium, taking a scenario containing specific parameters and data as an example.

[0080] like Figure 6 and Figure 7 As shown, in a specific application scenario, the audio signal processing link runs on a portable real-time communication terminal, which simultaneously enables digital signal processing functions such as voice enhancement, echo suppression, equalization, and dynamic range control. The base sampling rate is 4.8 × 10⁻⁶. 4  The parameters are: samples / s, basic quantization precision of 16 bits, processing cycle time of 20 ms, total input buffer capacity of 8192 samples, total output buffer capacity of 8192 samples, baseline input buffer sample budget of 960 samples, maximum input buffer sample budget of 3840 samples, minimum playback level of 1200 samples, and target playback level of 2400 samples. (4.8×10) 4 At a base sampling rate of samples / s, 20 ms corresponds to 960 audio samples. Therefore, the baseline cycle input buffer sample budget of 960 samples can cover one basic processing cycle. The maximum cycle input buffer sample budget is set to 3840 samples, with a corresponding buffer coverage time of 80 ms, which can provide buffer margin for the processing end when the load increases.

[0081] In S1, if the audio link load level of the previous processing cycle was the second load level, the delay status flag was normal, the output buffer level was the second level, the resource level execution result was complete, and the current processing cycle number was 128, then both the acquisition end and the processing end use the same initial link configuration data and the aforementioned status data to generate status input data. The status input data is concatenated according to the field order of "initial link configuration data, current processing cycle number, audio link load level, delay status flag, output buffer level, and resource level execution result." Because the acquisition end and the processing end use the same field order and the same deterministic verification operation, they can obtain the same processing cycle collaborative status code. The processing cycle collaborative status code is used in the current processing cycle to subsequently determine the processing end's resource execution level and the cycle input buffer sample budget, enabling the scheduling of the current processing cycle to inherit the confirmed running status of the previous processing cycle.

[0082] In S2, assume the number of active channels occupied in the current processing cycle is 4, the maximum number of active channels allowed in the same audio signal processing link is 8, the number of activated audio effect processing units is 3, the maximum number of audio effect processing units that can be activated simultaneously at the processing end is 6, the number of playable samples in the output buffer is 1800, the target playback level is 2400 samples, the number of historical processing cycles used for trend statistics is 10, the number of times the buffer coverage insufficiency time increased compared to the previous processing cycle in the last 10 confirmed processing cycles is 1, and the number of times the input buffer occupancy exceeded the input buffer occupancy boundary is 1. Substitute the above data into the expression in S22. Calculations yielded Further obtained Therefore, the audio link load adjustment amount for the current processing cycle is 0.7. If the processing end resource level boundary data corresponds to the first range of 0 to 0.45, the second range of greater than 0.45 and not greater than 0.75, and the third range of greater than 0.75 and not greater than 1, then the audio link load adjustment amount of 0.7 for the current processing cycle falls into the second range, and the processing end resource execution level is determined to be the second resource execution level.

[0083] In S3, if the processing end determines the periodic input buffer sample budget for the current processing cycle to be 2400 samples based on the second resource execution level, output buffer level, and audio link load adjustment, then the periodic input buffer sample budget needs to be converted into buffer coverage time. , Substitute the expression in S32, and first process the actual delay. Perform calculations Calculations yielded Therefore, when the actual processing latency is 0.046 s, the cache coverage shortage duration for the current processing cycle is 0 s, and the current processing cycle enters normal processing.

[0084] In another processing cycle, if the cycle input buffer sample budget remains 2400 samples and the base sampling rate remains 4.8 × 10⁻⁶, then... 4 If the actual processing delay recorded by the processing end is 0.062 s, then... , , Substitute the expression in S32 Calculations yielded Therefore, when the actual processing latency is 0.062 s, the cache coverage insufficiency duration for the current processing cycle is 0.012 s. If the first insufficiency boundary is 0.008 s and the second insufficiency boundary is 0.02 s, then 0.012 s exceeds the first insufficiency boundary but does not exceed the second insufficiency boundary, and the current processing cycle enters a mild cache coverage insufficiency processing stage.

[0085] In another processing cycle, if the cycle input buffer sample budget remains 2400 samples and the base sampling rate remains 4.8 × 10⁻⁶, then... 4 If the actual processing delay recorded by the processing end is 0.076 s, then... , , Substitute the expression in S32 Calculations yielded Therefore, when the actual processing latency is 0.076 s, the buffer coverage insufficiency duration for the current processing cycle is 0.026 s. If the second insufficiency boundary is 0.02 s, then 0.026 s exceeds the second insufficiency boundary, and the current processing cycle enters severe buffer coverage insufficiency processing. At this time, the acquisition end pauses writing new audio samples to the input buffer, the processing end prioritizes emptying the audio frames already in the input buffer, the playback output end outputs a protection audio frame, and resumes processing cycle scheduling after the recovery conditions are met.

[0086] The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the specific details of the above embodiments. Within the scope of the technical concept of the present invention, various simple modifications can be made to the technical solution of the present invention, and these simple modifications all fall within the protection scope of the present invention.

[0087] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the present invention will not describe the various possible combinations separately.

[0088] Furthermore, various different embodiments of the present invention can be combined in any way, as long as they do not violate the spirit of the present invention, they should also be regarded as the content disclosed by the present invention.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A method for collaborative processing of audio signal data, characterized in that, The methods include: Obtain the basic data of the cycle coordination of the same audio signal processing link, as well as the audio link load level, delay status flag, output buffer level and resource level execution result of the previous processing cycle, and generate the processing cycle coordination status code of the current processing cycle. Based on the number of effective channels occupied, the number of activated audio processing units, the number of playable samples in the output buffer, the number of times the buffer coverage is insufficient and the number of times the input buffer goes out of bounds, the audio link load adjustment amount is obtained, and the processing end resource execution level is determined by combining the processing cycle coordination status code. The periodic input buffer sample budget is determined based on the audio link load adjustment, the processing end resource execution level, and the output buffer level. The periodic input buffer sample budget is converted into buffer coverage duration according to the basic sampling rate. The insufficient buffer coverage duration is obtained based on the buffer coverage duration and the actual processing latency. Based on the duration of insufficient cache coverage, normal processing, mild insufficient cache coverage processing, or severe insufficient cache coverage processing is performed, and the processing cycle scheduling is resumed after the recovery conditions are met.

2. The audio signal data collaborative processing method according to claim 1, characterized in that, The processing cycle co-location status code for the current processing cycle includes: Read the same initial link configuration data, current processing cycle number, audio link load level of the previous processing cycle, delay status flag of the previous processing cycle, output buffer level of the previous processing cycle, and resource level execution result of the previous processing cycle. The initial link configuration data, the current processing cycle number, the audio link load level of the previous processing cycle, the delay status flag of the previous processing cycle, the output buffer level of the previous processing cycle, and the resource level execution result of the previous processing cycle are concatenated into status input data according to the same field order. The audio link load level, delay status flag, output buffer level, and resource level execution result of the previous processing cycle in the status input data are encoded to obtain the status field; a deterministic verification operation is performed on the status input data to obtain the verification field; the status field and verification field are combined according to the preset field order and mapped to the range of values ​​of the processing cycle collaborative status code to obtain the processing cycle collaborative status code of the current processing cycle.

3. The audio signal data collaborative processing method according to claim 1, characterized in that, Obtaining audio link load balancing includes: The channel occupancy ratio is obtained based on the number of active channels and the maximum number of active channels. The percentage of audio processing units used is obtained based on the number of enabled audio processing units and the maximum number of audio processing units. The degree of insufficiency of the output buffer is determined based on the number of playable samples in the output buffer and the target playback level. The percentage increase in cache coverage is obtained based on the number of times the duration of insufficient cache coverage increased relative to the previous processing cycle in the most recent confirmed processing cycles. The input cache out-of-bounds ratio is obtained based on the number of times the input cache usage exceeds the input cache usage boundary in the most recent confirmed processing cycles. The maximum value among the channel occupancy rate, the audio effect processing unit occupancy rate, and the output buffer level insufficiency is selected as the main load value. The audio link load adjustment amount is obtained based on the main load value, the buffer coverage insufficiency increase rate, and the input buffer out-of-bounds rate.

4. The audio signal data collaborative processing method according to claim 1, characterized in that, Determining the execution level of the processing resources includes: When the audio link load adjustment is within the first range and there is no insufficient cache coverage in the previous processing cycle, it is determined to be the first resource execution level; When the audio link load adjustment is in the second range, or when a slight buffer overlay occurs in the previous processing cycle, it is determined to be the second resource execution level; When the audio link load adjustment is in the third range, or when a severe cache overwrite occurred in the previous processing cycle and the recovery conditions are met, it is determined to be the third resource execution level; Among them, the processing end resource execution level is used to determine the processing thread budget, the sound effect processing unit execution budget, the data transfer priority, and the upper limit of the working frequency.

5. The audio signal data collaborative processing method according to claim 1, characterized in that, Determining the periodic input buffer sample budget includes: When the processing end resource execution level is the first resource execution level and the output cache level is the second or third level, the periodic input cache sample budget is determined according to the baseline periodic input cache sample budget. When the processing end resource execution level is the second resource execution level, the periodic input buffer sample budget is determined based on the baseline periodic input buffer sample budget and the sample increment corresponding to the audio link load adjustment amount; When the processing end resource execution level is the third resource execution level, or the output cache level is the first level, increase the periodic input cache sample budget; When the output buffer water level is at the third water level, the growth rate of the periodic input buffer sample budget is limited. Among them, the periodic input cache sample budget is not lower than the baseline periodic input cache sample budget, and not higher than the maximum periodic input cache sample budget.

6. The audio signal data collaborative processing method according to claim 1, characterized in that, The duration of insufficient cache coverage includes: Based on the periodic input cache sample budget and the base sampling rate, the periodic input cache sample budget is converted into cache coverage duration; When the audio frame corresponding to the current processing cycle is written to the input buffer, the input completion time is recorded; Record the output completion time when the audio frame has completed digital signal processing and been written to the output buffer; The actual processing delay is obtained based on the time interval between the output completion time and the input completion time; The actual processing latency is compared with the cache coverage time using the same units, and the time when the actual processing latency exceeds the cache coverage time is defined as the cache coverage insufficiency time; when the actual processing latency does not exceed the cache coverage time, the cache coverage insufficiency time is defined as zero.

7. The audio signal data collaborative processing method according to claim 1, characterized in that, Depending on the duration of insufficient cache coverage, the following processing methods are performed: normal processing, mild insufficient cache coverage processing, or severe insufficient cache coverage processing: When the buffer coverage is insufficient for a duration that does not exceed the first insufficient boundary, the input buffer sample budget is written into the audio sample according to the current processing cycle, the digital signal processing is completed according to the processing end resource execution level of the current processing cycle, and the delay status flag is recorded as normal. When the buffer coverage is insufficient for longer than the first insufficient boundary but not the second insufficient boundary, stop writing new audio samples to the input buffer area of ​​the current processing cycle, transfer the subsequently collected audio samples to the input buffer area or temporary holding area corresponding to the next processing cycle, transfer the audio frames that have not been processed to the waiting queue of the next processing cycle, output the short-term holding frame or spare audio frame at the end of the previous processing cycle, and record the delay status flag as a mild buffer coverage insufficient state. When the cache coverage is insufficient for longer than the second insufficient boundary, the writing of new audio samples to the input cache is paused, the audio frames already in the input cache are drained first, the receiving of new input cache write requests is paused, the operation utilization rate, input cache utilization rate, output cache level, processing thread blocking status and memory usage status are detected, the protection audio frame is output, and the delay status flag is recorded as a severe cache coverage insufficient state. The recovery conditions include the computational utilization rate being lower than the computational utilization rate boundary, the input cache utilization rate being lower than the input cache utilization rate boundary, the output cache level being the second or third level, and no processing thread blocking state being detected.

8. An audio signal data collaborative processing system, characterized in that, include: The periodic status generation module is used to obtain the periodic coordination basic data of the same audio signal processing link, as well as the audio link load level, delay status flag, output buffer level and resource level execution result of the previous processing cycle, and generate the processing cycle coordination status code of the current processing cycle. The resource level determination module is used to obtain the audio link load adjustment amount based on the number of effective channels occupied, the number of activated audio effect processing units, the number of playable samples in the output buffer, the number of times the buffer coverage is insufficient and the number of times the input buffer goes out of bounds, and to determine the resource execution level of the processing end in combination with the processing cycle collaborative status code. The cache budget determination module is used to determine the periodic input cache sample budget based on the audio link load adjustment, the processing end resource execution level, and the output cache level. The periodic input cache sample budget is converted into cache coverage duration according to the basic sampling rate, and the insufficient cache coverage duration is obtained based on the cache coverage duration and the actual processing latency. The hierarchical link protection module is used to perform normal processing, mild cache coverage processing, or severe cache coverage processing based on the duration of insufficient cache coverage, and to resume the processing cycle scheduling after the recovery conditions are met.

9. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory for implementing the audio signal data collaborative processing method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the audio signal data collaborative processing method according to any one of claims 1 to 7.

Images