Audio buffer management method and system based on data volume self-adaption

Abstract

The invention relates to the technical field of audio processing, in particular to an audio buffer area management method and system based on data volume self-adaption, and the method comprises the steps: determining a processing strategy according to the data volume of an audio buffer area, the processing strategy comprises the steps of directly executing data removal processing, judging whether to execute the data removal processing or not based on the fragment difference characterization value and not needing to execute the data removal processing; in the data removal processing, the removal data volume is determined based on the comprehensive evaluation value, and whether reduction adjustment is carried out on the removal data volume based on the key audio frame proportion is determined based on the removal influence coefficient; and determining a data removal strategy as a direct deletion strategy or an exchange strategy according to the residual data volume and the key frame data volume. The real-time processing capability of the system can be improved.

Description

Technical Field

[0001] This invention relates to the field of audio processing technology, and in particular to an audio buffer management method and system based on adaptive data volume. Background Technology

[0002] In audio processing systems, audio buffers are used to temporarily store audio data to ensure continuous playback and processing. However, as the amount of audio data changes, traditional audio buffer management methods often employ fixed processing strategies, failing to adapt to the actual data volume. This leads to frequent and costly data removal operations when the data volume is large, impacting the system's real-time processing performance. Therefore, reducing the overhead of data removal operations and improving the system's real-time processing capabilities has become a pressing technical problem for those skilled in the art.

[0003] Chinese Patent Publication No. CN120596052A discloses a dynamic audio buffer management method, system, and medium. The method includes: pre-setting a whitelist library of multiple application scenarios; matching the package name of the currently running foreground application with the whitelist library to determine the current application scenario; acquiring system performance status parameters in real time, including at least one of CPU utilization, memory utilization, and temperature control frequency limiting status; generating a buffer adjustment variable based on the statistical results of audio stuttering times; inputting the current application scenario, system performance status parameters, and buffer adjustment variable into a preset strategy model, and outputting a buffer level value; and dynamically adjusting the buffer size of the audio driver layer according to the level value to adapt to changes in system performance. It is evident that the above technical solution has the following problems: it does not formulate a differentiated data removal processing strategy for the audio buffer data itself, and the management and optimization dimensions of the audio buffer data are singular, resulting in poor utilization efficiency and anomaly handling efficiency of the audio buffer data. Summary of the Invention

[0004] To address this, the present invention provides an audio buffer management method and system based on adaptive data volume, which overcomes the problems in the prior art where there is no differentiated data removal processing strategy for the audio buffer data itself, the management and optimization of audio buffer data is only one dimension, resulting in poor utilization efficiency and anomaly handling efficiency of audio buffer data.

[0005] To achieve the above objectives, the present invention provides an audio buffer management method based on adaptive data volume, comprising: The processing strategy is determined based on the amount of data in the audio buffer. The processing strategies include directly performing data removal processing, determining whether to perform data removal processing based on the fragmentation-induced anomaly characterization value, and not performing data removal processing. The fragmentation anomaly characterization value is determined by selecting read / write latency jitter rate or audio frame timing correlation deviation based on the storage fragmentation rate determination result. In the data removal process, the amount of data to be removed is determined based on the comprehensive evaluation value, and the removal impact coefficient is used to determine whether to reduce the amount of data to be removed based on the proportion of key audio frames. Among them, the fragmentation influence coefficient of the affected area corresponding to the storage frame determines whether the storage frame is a key audio frame based on the inter-frame feature difference rate and the proportion of effective features or based on the proportion of effective features. The data removal strategy is determined based on the amount of remaining data and the amount of keyframe data: either a direct deletion strategy or a swap strategy.

[0006] Furthermore, for audio buffers with a data volume greater than or equal to a first preset data volume, data removal processing is performed directly; For audio buffers with a data volume greater than or equal to the second preset data volume and less than the first preset data volume, determine whether to perform data removal processing based on the fragmentation anomaly characterization value. For audio buffers with a data size smaller than the second preset data size, no data removal processing is required; The first preset data volume is greater than the second preset data volume.

[0007] Furthermore, data removal processing is performed on audio buffers whose fragmentation-induced distortion values ​​are greater than or equal to preset fragmentation-induced distortion values.

[0008] Furthermore, for audio buffers with a storage fragmentation rate greater than or equal to a preset storage fragmentation rate, the fragmentation-induced anomaly characterization value is determined based on the read / write latency jitter rate; For audio buffers with a storage fragmentation rate less than a preset storage fragmentation rate, the fragmentation-induced anomaly characterization value is determined based on the audio frame timing correlation deviation.

[0009] Furthermore, the amount of data to be removed is determined based on a comprehensive evaluation value; The amount of data removed is positively correlated with the overall evaluation value.

[0010] Furthermore, for audio buffers where the removal impact coefficient is greater than or equal to the preset removal impact coefficient, the amount of data to be removed is reduced based on the proportion of key audio frames; The decrease in the amount of removed data is positively correlated with the proportion of key audio frames.

[0011] Furthermore, for storage frames whose fragment impact coefficient in the affected area is greater than or equal to the preset fragment impact coefficient, whether they are key audio frames is determined based on the inter-frame feature difference rate and the proportion of effective features. For storage frames whose fragment impact coefficient in the affected area is less than the preset fragment impact coefficient, whether they are key audio frames is determined based on the proportion of effective features.

[0012] Furthermore, for audio buffers where the remaining data volume is greater than or equal to the preset remaining data volume or the key frame data volume is greater than or equal to the preset key frame data volume, the data removal strategy is determined to be a swap strategy.

[0013] Furthermore, for audio buffers where the remaining data volume is less than the preset remaining data volume and the key frame data volume is less than the preset key frame data volume, the data removal strategy is determined to be a direct deletion strategy.

[0014] This invention also provides an audio buffer management system based on adaptive data volume, comprising: The removal analysis module is used to determine the processing strategy based on the amount of data in the audio buffer. The processing strategies include directly performing data removal processing, determining whether to perform data removal processing based on the fragmentation-induced anomaly characterization value, and not performing data removal processing. The fragmentation anomaly characterization value is determined by selecting read / write latency jitter rate or audio frame timing correlation deviation based on the storage fragmentation rate determination result. The removal processing module, which is connected to the removal analysis module, is used to determine the amount of data to be removed based on the comprehensive evaluation value during the data removal process, and to determine whether to reduce the amount of data to be removed based on the proportion of key audio frames based on the removal impact coefficient. Among them, the fragmentation influence coefficient of the affected area corresponding to the storage frame determines whether the storage frame is a key audio frame based on the inter-frame feature difference rate and the proportion of effective features or based on the proportion of effective features. The strategy selection module, which is connected to the removal processing module, is used to determine the data removal strategy as either a direct deletion strategy or a swap strategy based on the remaining data volume and the keyframe data volume.

[0015] Compared with the prior art, the beneficial effects of the present invention are that, in the technical solution of the present invention, the amount of data in the audio buffer effectively reflects the cache accumulation state of audio data and the load of system audio processing, and then determines the processing strategy based on the amount of data in the audio buffer. This is conducive to targeted and refined management and differentiated processing of audio buffer data, thereby improving the rationality and efficiency of audio data processing, avoiding problems such as audio stuttering and delay caused by invalid data accumulation, and ensuring the smoothness and stability of audio playback.

[0016] Furthermore, this invention uses fragmentation-induced anomaly representation values ​​to reflect the overall anomaly risk level of the audio buffer caused by storage fragmentation. It can accurately characterize the read / write latency anomaly risk under high fragmentation and the audio frame timing disorder risk under low fragmentation based on the differences in storage fragmentation levels. Then, it determines whether to perform data removal processing based on the fragmentation-induced anomaly representation values. This is beneficial for accurately identifying and differentiating storage fragmentation anomaly risks, avoiding the loss of core audio data caused by blindly performing data removal processing, and avoiding problems such as buffer storage congestion and operational anomalies caused by failure to remove data in high-risk fragmentation states in a timely manner. This is beneficial for improving the pertinence and rationality of audio buffer data removal processing.

[0017] Furthermore, the present invention determines the amount of data to be removed based on a comprehensive evaluation value, which can integrate the degree of data accumulation in the audio buffer with the overall abnormal risk level caused by storage fragmentation to achieve a precise match between the amount of data to be removed and the actual storage state of the buffer. This effectively avoids the problems of storage congestion and continuous accumulation of fragmentation abnormal risks caused by insufficient removal. Based on the removal impact coefficient, it determines whether to reduce the amount of data to be removed based on the proportion of key audio frames. First, the removal impact coefficient is used to accurately determine the actual impact of the selected frames on the effective reading operation of audio data. Then, the reduction adjustment is made in combination with the proportion of key audio frames to avoid poor parsing and utilization efficiency of subsequent audio data processing due to data removal, thus taking into account both the buffer storage optimization needs and the effectiveness of subsequent audio data processing.

[0018] Furthermore, in this invention, the fragmentation impact coefficient of the affected region corresponding to the storage frame effectively reflects the fragmentation level of the storage environment of a single storage frame, quantitatively characterizes the degree of interference risk of fragmentation in this region to audio frame feature extraction and judgment, and then adaptively selects judgment parameters based on the fragmentation impact coefficient to determine whether the storage frame is a key audio frame. This is beneficial for implementing differentiated key audio frame judgment strategies for storage environments with different levels of fragmentation interference, improving the dynamic balance between key audio frame judgment accuracy and buffer management efficiency, and thus providing a precise judgment basis for subsequent fine-tuning of the amount of data removed based on the proportion of key audio frames.

[0019] Furthermore, this invention effectively reflects the remaining storage capacity of the audio buffer and the retention capacity of core audio frames after data removal by using the remaining data volume and keyframe data volume. It comprehensively characterizes the overall storage status and core audio data retention capability after buffer removal. Then, different data removal strategies are adaptively selected according to the remaining data volume and keyframe data volume. This is beneficial for implementing differentiated data removal operations for different remaining storage statuses and keyframe retention conditions of the buffer, achieving a precise balance between storage optimization efficiency and core audio data retention, and ensuring the continuity and effectiveness of subsequent audio data reading and processing. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the audio buffer management method based on adaptive data volume according to the present invention; Figure 2 This is a flowchart illustrating the present invention's method for determining whether to adjust the amount of removed data based on the proportion of key audio frames, using the removal impact coefficient as the basis for determining the reduction of the removed data volume. Figure 3 This is a flowchart illustrating how the present invention determines a data removal strategy based on the remaining data volume and the keyframe data volume. Figure 4 This is a module connection diagram of the audio buffer management system based on adaptive data volume of the present invention. Detailed Implementation

[0021] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0022] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0023] Please see Figures 1 to 3 As shown, this invention provides an audio buffer management method based on adaptive data volume, comprising: The processing strategy is determined based on the amount of data in the audio buffer. The processing strategies include directly performing data removal processing, determining whether to perform data removal processing based on the fragmentation-induced anomaly characterization value, and not performing data removal processing. The fragmentation anomaly characterization value is determined by selecting read / write latency jitter rate or audio frame timing correlation deviation based on the storage fragmentation rate determination result. In the data removal process, the amount of data to be removed is determined based on the comprehensive evaluation value, and the removal impact coefficient is used to determine whether to reduce the amount of data to be removed based on the proportion of key audio frames. Among them, the fragmentation influence coefficient of the affected area corresponding to the storage frame determines whether the storage frame is a key audio frame based on the inter-frame feature difference rate and the proportion of effective features or based on the proportion of effective features. The data removal strategy is determined based on the amount of remaining data and the amount of keyframe data: either a direct deletion strategy or a swap strategy.

[0024] This invention is applicable to all audio processing scenarios that have a clear need for real-time reading and writing of audio data and low-latency transmission, and are prone to storage fragmentation due to discontinuous audio reading and writing, data removal, operation interruption, etc. It can accurately solve the pain points of audio buffer fragmentation management and low-impact management.

[0025] The audio buffer is a dedicated hardware / software storage area used for temporarily caching audio frames in the audio data acquisition, transmission, processing, and playback chain. An audio frame is the smallest independent unit for digital audio time-domain processing and storage. It is a discrete data block formed by truncating continuous analog audio signals according to fixed specifications (frame length 10ms, sampling rate 48kHz, bit depth 16bit) and converting them to digital data. Each audio frame corresponds to audio data, which is an ordered set of all audio sampling points arranged sequentially within the frame. Each sampling point is mapped to the instantaneous vibration amplitude of the audio signal at the corresponding sampling time. The interval between any two adjacent audio sampling points = 1 / sampling rate. In this embodiment, the interval between two adjacent audio sampling points = (1 / 48000) s. The present invention applies a buffer management cycle, the duration of a single buffer management cycle is 2 minutes, and the user determines the amount of data in the audio buffer once at the end of a single buffer management cycle.

[0026] Specifically, for audio buffers with a data size greater than or equal to a first preset data size, data removal processing is performed directly. For audio buffers with a data volume greater than or equal to the second preset data volume and less than the first preset data volume, determine whether to perform data removal processing based on the fragmentation anomaly characterization value. For audio buffers with a data size smaller than the second preset data size, no data removal processing is required; The first preset data volume is greater than the second preset data volume.

[0027] Specifically, the amount of data in the audio buffer is the actual memory space occupied by the audio frame stored in the buffer at the end of the current buffer management cycle, in bits; The values ​​of the first preset data volume and the second preset data volume can be determined by the user according to the actual application scenario. The higher the user's demand for low latency and high real-time processing of the audio buffer, the smaller the values ​​of the first preset data volume and the second preset data volume will be. In this embodiment, the first preset data volume is 70% of the maximum memory volume of the audio buffer, and the second preset data volume is 40% of the maximum memory volume of the audio buffer.

[0028] Specifically, data removal processing is performed on audio buffers whose fragmentation-induced distortion values ​​are greater than or equal to preset fragmentation-induced distortion values.

[0029] When determining whether to perform data removal processing based on the fragmentation-induced anomaly characterization value, no data removal processing needs to be performed on audio buffers whose fragmentation-induced anomaly characterization value is less than the preset fragmentation-induced anomaly characterization value. The user can determine the value of the preset fragmentation anomaly characterization value based on the actual application scenario of the audio buffer and the identification and control requirements of storage fragmentation anomaly risk. The fragmentation anomaly characterization value reflects the overall anomaly risk level of the audio buffer caused by storage fragmentation. The greater the user's requirement for the accuracy of identification and judgment of storage fragmentation anomaly risk in the audio buffer, the smaller the value of the preset fragmentation anomaly characterization value. In this embodiment, the preset fragmentation anomaly characterization value is 0.5.

[0030] Specifically, for audio buffers with a storage fragmentation rate greater than or equal to a preset storage fragmentation rate, the fragmentation-induced anomaly characterization value is determined based on the read / write latency jitter rate. For audio buffers with a storage fragmentation rate less than a preset storage fragmentation rate, the fragmentation-induced anomaly characterization value is determined based on the audio frame timing correlation deviation.

[0031] Storage fragmentation rate = Total capacity of all storage fragments in the audio buffer at the end of the current buffer management cycle / Amount of memory used by the audio buffer at the end of the current buffer management cycle; Audio buffer storage fragments, measured in bits, are formed within the physical storage area of ​​the audio buffer by non-contiguous read / write, removal, or operation interruptions. The capacity of a single storage fragment is less than the fragment capacity threshold, and there is free storage space for audio data at both ends of the physical storage address. The fragment capacity threshold is set based on the smallest storage unit of a single audio frame to ensure that fragments cannot carry a complete audio frame, avoiding fragmentation from occupying effective storage resources and interfering with continuous frame read / write. Fragment capacity threshold = 48000 sampling rate × 0.01s × 16bit = 7680bit. The preset storage fragmentation rate can be determined by the user based on the actual application scenario. It can be understood that the storage fragmentation rate effectively reflects the degree of physical storage fragmentation in the audio buffer. When the storage fragmentation rate is high, it indicates that the storage fragmentation of the audio buffer has reached a significant level. Even with sufficient total free space, it is impossible to store consecutive audio frames, leading to large fluctuations in read / write operation latency and a significant increase in overall read / write latency. Determining the fragmentation-induced anomaly characterization value based on the read / write latency jitter rate can accurately characterize the degree of read / write latency anomalies caused by insufficient storage continuity in the audio buffer under high fragmentation conditions, effectively reflecting the storage anomaly risk in this scenario. When the storage fragmentation rate is low, the fragmentation level is low. At this time, storage fragmentation will not significantly interfere with the continuous allocation of audio data storage space or the continuous execution chain of read / write operations. The impact of fragmentation on read / write latency is weakened, but in this low-fragmentation state, non-continuous read / write and removal operations can still cause relative misalignment of the analysis frames between the storage timing sequence and the physical storage address sequence. This misalignment can further propagate to the audio data access stage, causing timing mismatch deviation between the reference audio frame and the reference analysis frame during the read operation. This makes the secondary timing correlation disorder caused by fragmentation the core cause of buffer storage anomalies in this state. Determining the fragmentation-induced anomaly characterization value based on the audio frame timing correlation deviation can more accurately characterize the anomaly risk of the audio buffer caused by storage timing problems in the low-fragmentation state. Therefore, the greater the user's requirement for the accuracy of audio buffer storage anomaly risk identification and control, the smaller the value of the preset storage fragmentation rate. In this embodiment, the preset storage fragmentation rate is 0.15.

[0032] Read / write latency jitter rate = standard deviation of latency corresponding to each valid operation in the current buffer management cycle / preset standard deviation of latency × first weighting coefficient + average latency corresponding to each valid operation in the current buffer management cycle / preset average latency × second weighting coefficient. The latency corresponding to a single valid operation is the duration between the operation completion time and the operation initiation time, in milliseconds; both the first and second weighting coefficients are 0.5; all successfully completed audio write / read operations in the current buffer management cycle are recorded as valid operations. The preset standard deviation and preset average delay values ​​can be determined based on the real-time requirements for audio data reading and writing. The greater the user's real-time requirements for audio data reading and writing, the smaller the preset standard deviation and preset average delay values ​​will be. In this embodiment, the preset standard deviation is 3ms and the preset average delay is 10ms.

[0033] When determining the fragmentation-induced anomaly characterization value based on the read / write latency jitter rate, the fragmentation-induced anomaly characterization value = read / write latency jitter rate; Audio frame timing correlation deviation = timing deviation rate / preset timing deviation rate × third weighting coefficient + matching deviation rate / preset matching deviation rate × fourth weighting coefficient, where both the third and fourth weighting coefficients are 0.5; The timing deviation rate is defined as the number of audio frames transmitted to the audio buffer and received during the current buffer management cycle, which are recorded as analysis frames. The timing deviation rate is calculated as the number of timing-misaligned analysis frames / the total number of analysis frames. The sequence of analysis frames ordered from first to last in the audio buffer is called the storage timing sequence. The sequence of analysis frames ordered from first to last in the physical storage address in the audio buffer is called the storage address sequence. The physical storage address is a fixed address identifier used to uniquely identify the actual physical storage location of each audio frame within the dedicated hardware storage area defined by the audio buffer. This address adopts a continuous ascending order encoding rule (starting from the initial storage location of the buffer, the address value increases sequentially as the storage location extends).

[0034] A misaligned analysis frame is an analysis frame whose order in the memory address sequence and memory timing sequence is different. The matching deviation rate is the average of the sub-matching deviation rates corresponding to each valid read operation; all successfully completed audio read operations in the current buffer management cycle are recorded as valid read operations; for a single valid read operation, the analysis frame already stored at the time when the valid read operation is initiated is recorded as the reference analysis frame, and the audio frame to be read in this valid read operation is recorded as the reference audio frame. The sub-matching deviation rate = 1 - (the number of reference audio frames in the storage address sequence and storage time sequence that have the same order as the reference analysis frame / the total number of reference audio frames); The preset timing deviation rate and preset matching deviation rate can be determined by the user according to the actual application scenario. The timing deviation rate and matching deviation rate effectively reflect the deviation status and potential abnormal risks of the timing association of audio frames in the audio buffer. The larger the timing deviation rate and matching deviation rate, the more likely the timing synchronization of audio frames and data reading and writing parsing are to be disordered and abnormal. The greater the user's demand for the accuracy of audio frame timing control and identification, the smaller the preset timing deviation rate and preset matching deviation rate will be. In this embodiment, the preset timing deviation rate is 0.3 and the preset matching deviation rate is 0.2.

[0035] When determining the fragmentation-induced anomaly characterization value based on the audio frame timing correlation deviation, the fragmentation-induced anomaly characterization value = audio frame timing correlation deviation.

[0036] Specifically, the amount of data to be removed is determined based on a comprehensive evaluation value; The amount of data removed is positively correlated with the overall evaluation value.

[0037] Specifically, the comprehensive evaluation value = data volume / second preset data volume × fifth weighting coefficient + fragmentation-induced anomaly characterization value / preset fragmentation-induced anomaly characterization value × sixth weighting coefficient, where both the fifth and sixth weighting coefficients are 0.5; The formula for calculating the amount of data to be removed, 'a', is: a = (overall evaluation value / preset overall evaluation value) × data removal threshold, where the data removal threshold is 20% of the maximum memory size of the audio buffer. The user can determine the preset comprehensive evaluation value based on the actual application scenario and the precise adjustment requirements of the amount of data removed from the audio buffer. The greater the user's need for precise control of the audio buffer storage status and fine adjustment of the amount of data removed, the smaller the preset comprehensive evaluation value will be. In this embodiment, the preset comprehensive evaluation value is 0.7.

[0038] Specifically, for audio buffers where the removal impact coefficient is greater than or equal to the preset removal impact coefficient, the amount of data to be removed is reduced based on the proportion of key audio frames; The decrease in the amount of removed data is positively correlated with the proportion of key audio frames.

[0039] Specifically, for audio buffers where the removal impact coefficient is less than the preset removal impact coefficient, there is no need to adjust the amount of data to be removed; The audio frame stored in the audio buffer at the end of the current buffer management cycle is recorded as the storage frame; Memory frames are selected in the order of the memory address sequence until the total capacity of the selected memory frames reaches 'a'. If the total capacity exceeds 'a' after selecting the current frame, the frame is discarded. The selected memory frames are recorded as selected frames, and the unselected memory frames are recorded as unselected frames. The removal impact coefficient is the average of the sub-impact coefficients corresponding to each valid read operation. The sub-impact coefficient corresponding to a single valid read operation = the number of selected frames read in that valid read operation / the number of storage frames read in that valid read operation. Key audio frame percentage = Number of key audio frames in the selected frames / Number of selected frames; When adjusting the amount of data to be removed based on the proportion of key audio frames, if the proportion of key audio frames is greater than or equal to the preset proportion of key audio frames, the reduction in the amount of data to be removed is 0.28a; if the proportion of key audio frames is less than the preset proportion of key audio frames, the reduction in the amount of data to be removed is 0.12a. The amount of data removed after adjusting for the reduction of the amount of data removed based on the proportion of key audio frames, determined by the removal impact coefficient, is denoted as a0, where a0 = a - the reduction value of the amount of data removed. Storage frames are selected in the order from first to last in the storage address sequence until the capacity of the selected storage frames reaches a0. If the total capacity exceeds a0 after selecting the current frame, the frame is discarded. The selected storage frames are denoted as the final selected frames, and the unselected storage frames are denoted as the final unselected frames. The preset removal influence coefficient can be determined by the user based on the accuracy requirements of effective audio data reading. It can be understood that the removal influence coefficient reflects the reading proportion of selected frames during effective audio data reading, directly demonstrating the degree of influence of selected frames on the effective audio data reading operation. The larger the removal influence coefficient, the higher the reading proportion of selected frames in each effective reading operation, and the stronger the influence of selected frames on the effective audio data reading operation. Therefore, the greater the user's accuracy requirements for effective audio data reading, the smaller the preset removal influence coefficient should be. In this embodiment, the preset removal influence coefficient is set to 0.45.

[0040] The preset key audio frame percentage can be determined by the user based on the actual application scenario. It can be understood that the key audio frame percentage can reflect the distribution characteristics of key audio frames in the selected frames, and intuitively reflect the degree of proportion of key audio frames in the audio data selection frames. The larger the key audio frame percentage, the higher the proportion of key audio frames in the selected frames, the stronger the influence of key audio frames on the effective reading of audio data, and the greater the user's demand for audio data integrity, the smaller the preset key audio frame percentage. In this embodiment, the preset key audio frame percentage is 0.4.

[0041] Specifically, for storage frames whose fragment impact coefficient in the affected area is greater than or equal to the preset fragment impact coefficient, whether they are key audio frames is determined based on the inter-frame feature difference rate and the proportion of effective features. For storage frames whose fragment impact coefficient in the affected area is less than the preset fragment impact coefficient, whether they are key audio frames is determined based on the proportion of effective features.

[0042] Specifically, for a single storage frame, using the physical storage address of the storage frame as a reference point, storage regions are selected on both sides of the reference point, with the capacity of each selected storage region being 2% of the maximum memory size of the audio buffer. The storage regions on both sides of the reference point together constitute the influence area corresponding to the single storage frame. If it is the starting frame of the buffer (physical storage address is the minimum value), only a storage region of 2% of the maximum memory size of the audio buffer is selected to the right of the reference point, and the area on one side is the influence area of ​​the frame. If it is the ending frame of the buffer (physical storage address is the maximum value), only a storage region of 2% of the maximum memory size of the audio buffer is selected to the left of the reference point, and the area on one side is the influence area of ​​the frame.

[0043] The fragmentation impact coefficient of the affected area corresponding to a single storage frame = the total capacity of storage fragments in the affected area / the capacity of the affected area corresponding to the storage frame, where the capacity is in bits; For a single memory frame, the memory frame is designated as the target memory frame, and the memory frames adjacent to the target memory frame in the influence area of ​​the target memory frame are designated as neighboring memory frames. The inter-frame feature difference rate corresponding to the target memory frame is the larger value among the sub-difference rates corresponding to each neighboring memory frame. The sub-difference rate corresponding to a single neighboring memory frame = the absolute value of the difference between the jump coefficients of the target memory frame and the neighboring memory frame / the larger value among the jump coefficients of the target memory frame and the neighboring memory frame. The method for determining the jump coefficient of a single storage frame is as follows: extract all audio sampling points of the audio data corresponding to the storage frame, determine the sign of the instantaneous vibration amplitude of adjacent sampling points one by one, and count the number of times that the instantaneous vibration amplitudes of adjacent sampling points have opposite signs (the number of times that the instantaneous vibration amplitudes of adjacent sampling points have the same sign or contain zero values ​​is not included in the count). The number of counts is the jump coefficient of a single storage frame. The larger the absolute value of the instantaneous vibration amplitude, the higher the vibration intensity of the audio signal at that moment. The sign of the vibration amplitude represents the vibration direction of the audio signal. The number of audio sampling points = sampling rate × frame length. Zero-padding is performed on the 480 audio sampling points of a single storage frame (adding 32 zero-value sampling points to the end of the sampling point sequence), bringing the total number of data points to 512. Then, a 512-point FFT is used to complete the frequency domain transformation, resulting in 256 frequency points and their corresponding amplitudes. Frequency points within the preset core frequency band [300Hz, 3400Hz] are recorded as effective frequency points. The effective feature percentage of a single storage frame = the sum of the amplitudes corresponding to each effective frequency point / (the sum of the amplitudes corresponding to each frequency point + α), where α is a correction coefficient, α = 0.001. The amplitude corresponding to a single frequency point is the amplitude value of the frequency domain component corresponding to each frequency point after the audio data of a single storage frame is converted to the frequency domain by the Fast Fourier Transform, representing the energy intensity of that frequency component in the audio signal of the current storage frame. The value of the preset fragmentation impact coefficient can be determined by the user based on the actual application scenario. The fragmentation impact coefficient reflects the density of storage fragments and the fragmentation level of the storage environment within the affected area corresponding to a single storage frame. When the fragmentation impact coefficient of the affected area is large, it indicates that the storage environment corresponding to the storage frame is significantly fragmented. High fragmentation will lead to a higher risk of interference with the data integrity and temporal continuity of audio frames, which will directly affect the authenticity of audio feature quantization. Introducing the inter-frame feature difference rate into the judgment can offset the feature distortion caused by fragmentation interference through the feature correlation verification between time-domain frames, thereby improving the accuracy and reliability of the key audio frame judgment results. When the fragmentation impact coefficient of the affected area is small, it indicates that the fragmentation level of the storage environment corresponding to the storage frame is slight and the interference risk is negligible. Judging based solely on the proportion of effective features can reduce the overall computational complexity of key audio frame judgment and improve judgment efficiency. The greater the user's demand for storage environment stability and the accuracy of key audio frame judgment, the smaller the value of the preset fragmentation impact coefficient. In this embodiment, the preset fragmentation impact coefficient is 0.4.

[0044] When determining whether a frame is a key audio frame based on the inter-frame feature difference rate and the proportion of effective features, the stored frames with an inter-frame feature difference rate greater than the preset inter-frame feature difference rate and a proportion of effective features greater than the preset proportion of effective features are recorded as key audio frames. When determining whether a frame is a key audio frame based on the proportion of effective features, the storage frames with a proportion of effective features greater than the preset proportion of effective features are recorded as key audio frames. The preset values ​​of the inter-frame feature difference rate and the preset effective feature ratio can be determined by the user based on the comprehensive requirements of the key audio frame judgment accuracy and audio signal processing efficiency in the actual application scenario. The inter-frame feature difference rate and the effective feature ratio effectively reflect the degree of difference in the temporal features of a single storage frame relative to its neighboring storage frames, as well as the proportion level of its own frequency domain core effective features. They comprehensively characterize the signal feature recognition degree and core audio information carrying capacity of the audio frame. The greater the user's requirements for the accuracy of key audio frame judgment and the precision of core audio signal screening, the larger the values ​​of the preset inter-frame feature difference rate and the preset effective feature ratio. In this embodiment, the preset inter-frame feature difference rate is 0.6 and the preset effective feature ratio is 0.5.

[0045] Specifically, for audio buffers where the remaining data volume is greater than or equal to the preset remaining data volume or the key frame data volume is greater than or equal to the preset key frame data volume, the data removal strategy is determined to be a swap strategy.

[0046] Remaining data size = audio buffer data size - a0; keyframe data size is the amount of memory occupied by keyframes in the final unselected frames; The preset values ​​for the remaining data volume and the preset keyframe data volume can be determined by the user based on the actual application scenario of the audio buffer and the overall processing efficiency control requirements after data removal. The remaining data volume and the keyframe data volume effectively reflect the remaining storage guarantee level of the audio buffer and the retention capacity of core key audio frames after data removal. The larger the values ​​of these two values, the more it will reduce the subsequent overall processing efficiency of the audio buffer. The greater the user's requirement for the precision of the overall processing efficiency after the removal of audio buffer data, the smaller the values ​​of the preset remaining data volume and the preset keyframe data volume will be. In this embodiment, the preset remaining data volume is 50% of the maximum memory volume of the audio buffer, and the preset keyframe data volume is 20% of the maximum memory volume of the audio buffer. In the swapping strategy, each final selected frame is deleted, a temporary buffer is created, each final unselected frame is copied to the temporary buffer, and then the temporary buffer is swapped with the audio buffer. The swapping method is used when swapping the temporary buffer with the audio buffer. The swapping method is an address / metadata swapping method that switches the memory mapping relationship between the two buffers and points the access entry of the original buffer to the temporary buffer.

[0047] Specifically, for audio buffers where the remaining data volume is less than the preset remaining data volume and the key frame data volume is less than the preset key frame data volume, the data removal strategy is determined to be a direct deletion strategy.

[0048] In the direct deletion strategy, each final selected frame is directly deleted.

[0049] Please see Figure 4 The diagram shown is a module connection diagram of the audio buffer management system based on adaptive data volume of the present invention. The present invention also provides an audio buffer management system based on adaptive data volume, comprising: The removal analysis module is used to determine the processing strategy based on the amount of data in the audio buffer. The processing strategies include directly performing data removal processing, determining whether to perform data removal processing based on the fragmentation-induced anomaly characterization value, and not performing data removal processing. The fragmentation anomaly characterization value is determined by selecting read / write latency jitter rate or audio frame timing correlation deviation based on the storage fragmentation rate determination result. The removal processing module, which is connected to the removal analysis module, is used to determine the amount of data to be removed based on the comprehensive evaluation value during the data removal process, and to determine whether to reduce the amount of data to be removed based on the proportion of key audio frames based on the removal impact coefficient. Among them, the fragmentation influence coefficient of the affected area corresponding to the storage frame determines whether the storage frame is a key audio frame based on the inter-frame feature difference rate and the proportion of effective features or based on the proportion of effective features. The strategy selection module, which is connected to the removal processing module, is used to determine the data removal strategy as either a direct deletion strategy or a swap strategy based on the remaining data volume and the keyframe data volume.

[0050] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. An audio buffer management method based on adaptive data volume, characterized in that, include: The processing strategy is determined based on the amount of data in the audio buffer. The processing strategies include directly performing data removal processing, determining whether to perform data removal processing based on the fragmentation-induced anomaly characterization value, and not performing data removal processing. The fragmentation anomaly characterization value is determined by selecting read / write latency jitter rate or audio frame timing correlation deviation based on the storage fragmentation rate determination result. In the data removal process, the amount of data to be removed is determined based on the comprehensive evaluation value, and the removal impact coefficient is used to determine whether to reduce the amount of data to be removed based on the proportion of key audio frames. Among them, the fragmentation influence coefficient of the affected area corresponding to the storage frame determines whether the storage frame is a key audio frame based on the inter-frame feature difference rate and the proportion of effective features or based on the proportion of effective features. The data removal strategy is determined based on the amount of remaining data and the amount of keyframe data: either a direct deletion strategy or a swap strategy.

2. The audio buffer management method based on adaptive data volume according to claim 1, characterized in that, For audio buffers with a data size greater than or equal to the first preset data size, data removal processing is performed directly; For audio buffers with a data volume greater than or equal to the second preset data volume and less than the first preset data volume, determine whether to perform data removal processing based on the fragmentation anomaly characterization value. For audio buffers with a data size smaller than the second preset data size, no data removal processing is required; The first preset data volume is greater than the second preset data volume.

3. The audio buffer management method based on adaptive data volume according to claim 2, characterized in that, Data removal processing is performed on audio buffers whose fragmentation-induced distortion values ​​are greater than or equal to preset fragmentation-induced distortion values.

4. The audio buffer management method based on adaptive data volume according to claim 3, characterized in that, For audio buffers with a storage fragmentation rate greater than or equal to a preset storage fragmentation rate, the fragmentation-induced anomaly characterization value is determined based on the read / write latency jitter rate. For audio buffers with a storage fragmentation rate less than a preset storage fragmentation rate, the fragmentation-induced anomaly characterization value is determined based on the audio frame timing correlation deviation.

5. The audio buffer management method based on adaptive data volume according to claim 1, characterized in that, The amount of data to be removed is determined based on a comprehensive evaluation value; The amount of data removed is positively correlated with the overall evaluation value.

6. The audio buffer management method based on adaptive data volume according to claim 2, characterized in that, For audio buffers where the removal impact coefficient is greater than or equal to the preset removal impact coefficient, the amount of data to be removed is reduced based on the proportion of key audio frames. The decrease in the amount of removed data is positively correlated with the proportion of key audio frames.

7. The audio buffer management method based on adaptive data volume according to claim 6, characterized in that, For storage frames whose fragment impact coefficient is greater than or equal to the preset fragment impact coefficient in the affected area, whether they are key audio frames is determined based on the inter-frame feature difference rate and the proportion of effective features. For storage frames whose fragment impact coefficient in the affected area is less than the preset fragment impact coefficient, whether they are key audio frames is determined based on the proportion of effective features.

8. The audio buffer management method based on adaptive data volume according to claim 1, characterized in that, For audio buffers where the remaining data volume is greater than or equal to the preset remaining data volume or the keyframe data volume is greater than or equal to the preset keyframe data volume, the data removal strategy is determined to be the swap strategy.

9. The audio buffer management method based on adaptive data volume according to claim 8, characterized in that, For audio buffers where the remaining data volume is less than the preset remaining data volume and the key frame data volume is less than the preset key frame data volume, the data removal strategy is determined to be a direct deletion strategy.

10. A management system applying the audio buffer management method based on adaptive data volume as described in any one of claims 1 to 9, characterized in that, include: The removal analysis module is used to determine the processing strategy based on the amount of data in the audio buffer. The processing strategies include directly performing data removal processing, determining whether to perform data removal processing based on the fragmentation-induced anomaly characterization value, and not performing data removal processing. The fragmentation anomaly characterization value is determined by selecting read / write latency jitter rate or audio frame timing correlation deviation based on the storage fragmentation rate determination result. The removal processing module, which is connected to the removal analysis module, is used to determine the amount of data to be removed based on the comprehensive evaluation value during the data removal process, and to determine whether to reduce the amount of data to be removed based on the proportion of key audio frames based on the removal impact coefficient. Among them, the fragmentation influence coefficient of the affected area corresponding to the storage frame determines whether the storage frame is a key audio frame based on the inter-frame feature difference rate and the proportion of effective features or based on the proportion of effective features. The strategy selection module, which is connected to the removal processing module, is used to determine the data removal strategy as either a direct deletion strategy or a swap strategy based on the remaining data volume and the keyframe data volume.

Images