Calculation method and device of audio output sampling rate, electronic equipment and storage medium

Abstract

The invention provides an audio output sampling rate calculation method and device, electronic equipment and a storage medium. The method is applied to an audio playing system, and comprises the following steps: in an audio data playing process of an audio buffer area, detecting a filling level of the audio buffer area according to a preset detection frequency; wherein the filling level is used for indicating the current data volume of the audio buffer area; based on the filling level, first clock skew corresponding to the recent multiple detection periods is obtained, and a clock skew set is obtained; calculating a first clock calibration amount according to a first clock skew corresponding to the current detection period in the clock skew set; performing clock drift trend prediction on the clock skew set, and determining a second clock calibration amount according to a prediction result; and according to the first clock calibration amount and the second clock calibration amount, determining the sampling rate of at least one frame of audio data output audio buffer in the future. The method and the device are used for carrying out real-time, accurate and self-adaptive clock calibration on audio playing.

Description

Technical Field

[0001] This disclosure relates to the field of audio processing technology, and in particular to a method, apparatus, electronic device, and storage medium for calculating audio output sampling rate. Background Technology

[0002] In the field of audio processing technology, audio signals (such as 5.1 channel PCM data from a Blu-ray player) are transmitted to audio processing devices (such as power amplifiers and sound cards) through interfaces such as HDMI and optical fiber at a fixed sampling rate (e.g., 48kHz, or 48,000 samples per second). After the transmitted data undergoes relevant processing (such as decoding, mixing, and sound effects), it is converted into an analog signal at another fixed sampling rate to drive the playback device to emit sound.

[0003] Due to the limitations of physical components, there are no two clock sources in the world that are completely identical in frequency. The errors mainly come from crystal oscillator accuracy, temperature drift, aging, etc. The mismatch between the input clock and the output clock will lead to inaccurate sampling rate conversion, resulting in audio playback phenomena such as popping, muting, or stuttering.

[0004] Traditional solutions primarily rely on hardware phase-locked loops (PLLs) for clock recovery. However, PLLs are slow to respond and prone to losing lock when the clock source quality is poor. On the software side, simple sample rate conversion can alleviate the problem to some extent, but it cannot fundamentally solve the clock domain mismatch issue. Especially during long-term operation, accumulated errors can still lead to audio synchronization problems. Therefore, there is an urgent need for a new method that can perform real-time, accurate, and adaptive clock calibration. Summary of the Invention

[0005] In view of this, the purpose of this disclosure is to provide a method, apparatus, electronic device and storage medium for calculating the audio output sampling rate for real-time, accurate and adaptive clock calibration of audio playback.

[0006] In a first aspect, embodiments of this disclosure provide a method for calculating an audio output sampling rate, applied to an audio playback system. The method includes: during the playback of audio data in an audio buffer, detecting the fill level of the audio buffer at a preset detection frequency; wherein the fill level is used to indicate the current data volume of the audio buffer; based on the fill level, obtaining a first clock deviation corresponding to the most recent multiple detection cycles to obtain a clock deviation set; wherein the first clock deviation is used to indicate the ratio of clock deviation between audio data input and output to the audio buffer within a unit time; calculating a first clock calibration amount according to the first clock deviation corresponding to the current detection cycle in the clock deviation set; performing clock drift trend prediction on the clock deviation set, and determining a second clock calibration amount based on the prediction result; and determining the sampling rate of at least one future frame of audio data output to the audio buffer based on the first clock calibration amount and the second clock calibration amount to obtain a target output sampling rate.

[0007] Secondly, embodiments of this disclosure provide an audio output sampling rate calculation device applied to an audio playback system. The device includes: a detection module, configured to detect the fill level of an audio buffer at a preset detection frequency during audio data playback; wherein the fill level is used to indicate the current data volume of the audio buffer; an acquisition module, configured to acquire first clock deviations corresponding to the most recent multiple detection cycles based on the fill level, thereby obtaining a clock deviation set; wherein the first clock deviation is used to indicate the ratio of clock deviation between audio data input and output to the audio buffer within a unit time; a calculation module, configured to calculate a first clock calibration amount based on the first clock deviation corresponding to the current detection cycle in the clock deviation set; a prediction module, configured to predict the clock drift trend of the clock deviation set and determine a second clock calibration amount based on the prediction result; and a determination module, configured to determine the sampling rate of at least one future frame of audio data output to the audio buffer based on the first clock calibration amount and the second clock calibration amount, thereby obtaining a target output sampling rate.

[0008] Thirdly, embodiments of this disclosure provide an electronic device, including a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor executes the machine-executable instructions to implement the above-described method for calculating the audio output sampling rate.

[0009] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions cause the processor to implement the above-described method for calculating the audio output sampling rate.

[0010] The embodiments disclosed herein bring the following beneficial effects: The aforementioned method, apparatus, electronic device, and storage medium for calculating the audio output sampling rate periodically monitor the fill level of the audio buffer to estimate the instantaneous deviation of the clock. For both instantaneous and long-term clock deviations, the calibration amount used to compensate for the instantaneous deviation and the calibration amount used to compensate for the long-term deviation are calculated. Then, the clock deviation is calibrated by adjusting the output sampling rate. This can fundamentally solve the problem of short-term and long-term mismatch in the clock domain, making clock calibration more real-time and accurate, and able to adapt to different deviation levels.

[0011] Other features and advantages of this disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the disclosure. The objects and other advantages of this disclosure are realized and obtained through the structures particularly pointed out in the description, claims and drawings.

[0012] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0014] Figure 1 This is a flowchart of one embodiment of the method for calculating the audio output sampling rate in this disclosure; Figure 2 A schematic diagram of an audio output sampling rate calculation device provided for an embodiment of this disclosure; Figure 3 This is a schematic diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0016] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0017] For ease of understanding, the specific process of the embodiments of this disclosure is described below. Please refer to [link / reference]. Figure 1 This disclosure applies to an audio playback system. One embodiment of the method for calculating the audio output sampling rate in this disclosure includes: Step S10: During the playback of audio data in the audio buffer, the fill level of the audio buffer is detected at a preset detection frequency; wherein, the fill level is used to indicate the current amount of data in the audio buffer. An audio playback system refers to a system with audio playback functionality, which may include an audio playback device. During the playback process of the audio playback device in the audio playback system, the method for calculating the audio output sampling rate provided in this disclosure embodiment can be executed to reduce the impact of clock differences on the audio playback effect and reduce the occurrence of phenomena such as popping, muting, or stuttering.

[0018] An audio buffer is used for data exchange and rate buffering between the input and output clock domains. The audio signal is input to the audio buffer at a specified input sampling rate, and then output to the audio buffer at a fixed output sampling rate for audio playback. The size of the audio buffer can be dynamically changed; specifically, it can be dynamically calculated based on a specified input sampling rate (a fixed value) and a preset detection frequency. No specific limitation is set here.

[0019] It's understandable that the fill level doesn't just refer to the absolute amount of data in the audio buffer, but rather to the deviation of the current effective data amount from the nominal center position of the audio buffer. This level reflects the instantaneous, physical measure of the difference between the input and output clock frequencies. The nominal center position of the buffer refers to the middle data position within the audio buffer. For example, if the audio buffer size is 2048 samples, then the nominal center position is 1024 samples; the specific value is not limited here.

[0020] In one implementation, the audio buffer is a First-In-First-Out (FIFO) circular buffer. Ideally, when the input and output clocks are perfectly matched, the audio buffer should maintain a 50% fill level. However, due to the mismatch between the input and output clocks, inaccurate sampling rate conversion can occur, resulting in an imbalance in the accumulation or consumption of audio data in the audio buffer. This implementation monitors the fill level of the audio buffer to calculate the degree of clock deviation, and then calibrates the clock deviation by adjusting the output sampling rate. This fundamentally solves the problem of clock domain mismatch, making clock calibration more real-time and accurate, and adaptable to different degrees of deviation.

[0021] In one implementation, the fill level can be expressed as the number of sample points, the number of bytes, or a percentage of the total buffer capacity. For example, for a buffer with a nominal center position of 4096 sample points, if 4119 sample points of stored data are detected in the current detection cycle, the current fill level can be expressed as "4119 sample points".

[0022] The preset detection frequency is used to periodically detect the fill level of the audio buffer. The preset detection frequency can be dynamically set according to the input sampling rate. For example, when the input sampling rate is 48kHz, the preset detection frequency can be 100ms, which means that the fill level of the audio buffer is detected once every 100ms.

[0023] Step S20: Based on the fill level, obtain the first clock deviation corresponding to the most recent multiple detection cycles to obtain a clock deviation set; wherein, the first clock deviation is used to indicate the proportion of clock deviation between the audio data input and output audio buffers in a unit time. In each detection cycle, the first clock deviation corresponding to that detection cycle can be calculated based on the fill level of the audio buffer detected in that detection cycle. This deviation represents the proportion of clock deviation between the audio data input and output audio buffers within a unit of time in that detection cycle. This quantization achieves an effective mapping from the buffer state to the clock frequency error.

[0024] In one implementation, the fill level of the audio buffer within a detection cycle can be normalized to convert the fill level into a relative clock frequency deviation per unit time, which can be expressed in parts per million (ppm), representing the deviation of the output clock from the input clock. The specific unit is not limited here. For example, assuming the first clock deviation of a detection cycle is 5 ppm, it means the output clock is five parts per million slower than the input clock, or 5 Hz slower for a 1 MHz master clock.

[0025] The clock skew set is a time-series queue used to record historical data of the first clock skew over a recent period (e.g., 50 detection cycles). The number of detection cycles in the clock skew set can be dynamically set, and is not limited here. For example, if the clock skew set records the first clock skew of the most recent 10 seconds, and the preset detection frequency is 100ms, then the clock skew set will contain the first clock skew corresponding to the most recent 100 detection cycles.

[0026] Step S30: Calculate the first clock calibration amount based on the first clock deviation corresponding to the current detection cycle in the clock deviation set; The first clock calibration value is an instantaneous correction value based on real-time feedback. It is a clock calibration value for the first clock deviation corresponding to the most recent detection period (i.e., the current detection period), used to correct the first clock deviation corresponding to the most recent detection period. In one implementation, the first clock deviation corresponding to the current detection period can be divided into multiple identical or different calibration values ​​according to a certain division rule / algorithm to obtain the first clock calibration value. The number of divisions can also be determined by the division rule / algorithm, and is not limited here.

[0027] The first clock calibration value can include the clock calibration value corresponding to each time period within a certain time period of clock calibration. Specifically, the time period of clock calibration is determined according to the playback frame rate of the audio data. For example, assuming that the first clock calibration value and the second clock calibration value need to be adjusted within the next 20 frames, then the time period of clock calibration is the duration of 20 frames. If the frame rate is 60 FPS, then this duration is 1 / 3s. The first clock calibration value contains 20 (corresponding to 20 frames) first clock calibration values, but the specific number is not limited here.

[0028] In one implementation, when calculating the first clock calibration amount, the current clock calibration intensity factor and the current clock calibration smoothing factor can be determined first based on the first clock deviation corresponding to the current detection period. Then, based on the current clock calibration intensity factor and the current clock calibration smoothing factor, the clock calibration amount for at least one frame of audio data in the future is calculated to obtain the second clock calibration amount. The current clock calibration intensity factor indicates the intensity of the clock calibration, enabling a balance between rapid convergence and minimal disturbance. The current clock calibration smoothing factor suppresses transient jitter and measurement noise, making the clock calibration gradual.

[0029] The current clock calibration intensity factor can be determined based on the absolute value of the first clock deviation corresponding to the current detection cycle. The current clock calibration smoothing factor can be determined based on the first clock deviation corresponding to the current detection cycle and the preset smoothing parameter. The specifics are not limited here.

[0030] Step S40: Predict the clock drift trend of the clock deviation set, and determine the second clock calibration amount based on the prediction result; The second clock calibration amount is a compensation amount used to preemptively offset future drift. Since the clock deviation set is input with new data in each detection cycle, by performing clock drift trend prediction on the clock deviation set in each detection cycle through this embodiment, the long-term inertia or trend of clock frequency drift can be identified, such as a continuous and slow frequency change trend caused by changes in ambient temperature or device aging, thereby fundamentally offsetting the clock deviation caused by physical devices.

[0031] When predicting clock drift trends, the time series of clock deviation sets can be predicted using the Autoregressive Integrated Moving Average (ARIMA) model to obtain the prediction results; the clock deviation sets can also be exponentially smoothed to obtain the processed results as the prediction results of clock drift trends; other trend prediction algorithms / models can also be used to obtain the prediction results of clock drift trends, which will not be elaborated here.

[0032] After obtaining the prediction results, the amount of clock deviation to be adjusted in each clock calibration period can be determined based on the clock calibration time period and the weight ratio of the prediction results, thus obtaining the second clock calibration amount. The clock calibration time period can be determined according to the playback frame rate of the audio data. For example, assuming the first and second clock calibration amounts need to be adjusted within the next 20 frames, then the clock calibration time period is the duration of 20 frames. If the frame rate is 60 FPS, then this duration is 1 / 3 second. The second clock calibration amount contains 20 (corresponding to 20 frames) second clock calibration amounts; the specific number is not limited here.

[0033] Step S50: Determine the sampling rate of the audio buffer for at least one frame of audio data output in the future based on the first clock calibration value and the second clock calibration value, and obtain the target output sampling rate.

[0034] Based on the first clock calibration value and the second clock calibration value, the target clock calibration value for each future frame of audio data can be calculated first. Then, based on the target clock calibration value for each frame of audio data, the output sampling rate of each frame of audio data can be calculated, thereby determining the sampling rate of the audio buffer for at least one future frame of audio data and obtaining the target output sampling rate.

[0035] When calculating the target clock calibration value, a weighted summation method can be used to obtain the target clock calibration value for each frame of audio data. Then, the target clock calibration value can be converted into a sampling rate to obtain the target output sampling rate.

[0036] The audio output sampling rate calculation method provided by the above embodiments periodically monitors the fill level of the audio buffer, calculates the instantaneous deviation of the clock, and calculates the calibration amount to compensate for the instantaneous deviation and the calibration amount to compensate for the long-term deviation for both instantaneous and long-term clock deviations. Then, the clock deviation is calibrated by adjusting the output sampling rate. This can fundamentally solve the problem of short-term and long-term clock domain mismatch, making clock calibration more real-time and accurate, and able to adapt to different deviation levels.

[0037] Next, we will explain the specific calculation method for the audio output sampling rate.

[0038] In one implementation, the step of obtaining a set of clock deviations by acquiring the first clock deviations corresponding to the most recent multiple detection cycles based on the fill level includes: calculating the difference between the fill level of the audio buffer detected in the current detection cycle and a preset baseline fill level to obtain a fill level deviation value corresponding to the current detection cycle; calculating the ratio of clock deviations between the audio data input and output audio buffers per unit time based on the fill level deviation value corresponding to the current detection cycle, the detection time interval of the preset detection frequency, and the current output sampling rate of the audio buffer to obtain the first clock deviation corresponding to the current detection cycle; acquiring the first clock deviations corresponding to the most recent multiple detection cycles in historical detection cycles to obtain a set of clock deviations; and the first clock deviation corresponding to the current detection cycle. for:

[0039] in, This indicates the fill level deviation value corresponding to the current testing cycle. The current output sampling rate of the audio buffer is represented by I, and the detection time interval of the preset detection frequency is represented by I.

[0040] The clock deviation set contains the first clock deviations corresponding to the most recent multiple detection cycles. The calculation method for the first clock deviation for each detection cycle is the same. In this embodiment, This represents the first clock deviation corresponding to the current detection period, where i represents the number of elements in the clock deviation set, which is the number of first clock deviations and equal to the number of detection periods. The elements in the clock deviation set are arranged in chronological order according to the detection periods. It also indicates the first clock deviation corresponding to the latest detection cycle. The first clock deviation corresponds to the previous detection cycle of the latest detection cycle, and so on. In this embodiment, i has the same meaning, and will not be described in detail here.

[0041] In this embodiment, represents the filling level deviation value corresponding to the current detection period, which is the difference between the filling level of the audio buffer detected in the current detection period and the preset reference filling level. For example, assuming that the preset reference filling level is 1022 sampling points and the filling level of the audio buffer detected in the current detection period is 1045 sampling points, then, sampling points, and specific values are not limited here.

[0042] In this embodiment, represents the current output sampling rate of the audio buffer, which can be the target output sampling rate calculated in the previous detection period or the output sampling rate detected currently, and specific values are not limited here.

[0043] In this embodiment, represents the number of sampling points that should theoretically be output within time I, Normalize the deviation value to the proportion of the theoretical output amount, that is, the relative deviation proportion of frequency per unit time, which can be used as the direct basis for clock correction.

[0044] For the (i - n)th (0 < n < i, n is a positive integer) detection period, the calculation method of the corresponding first clock deviation is the same, as shown below:

[0045] Among them, represents the filling level deviation value corresponding to the (i - n)th detection period, represents the output sampling rate of the audio buffer in the (i - n)th detection period, and I represents the detection time interval of the preset detection frequency.

[0046] The reference filling level can be an ideal value, such as the nominal center position of the audio buffer, that is, in the ideal case of clock synchronization, the filling level of the audio buffer; it can also be an arithmetic value, such as the average value, central value, variance, standard deviation, etc. of the filling level within a certain duration or in a specific state that can measure the reference filling level of the audio buffer, and specific values are not limited here.

[0047] In one embodiment, the above specific state may refer to that the audio playback system is in a stable playback state, or the first N seconds (N is a positive number) before the audio playback system starts playing audio. The reference filling level is used to indicate: the average filling level of the audio buffer within a certain duration when the audio playback system is in a stable playback state.

[0048] Specifically, the stable playback state can be determined based on the prediction result obtained from the clock drift trend prediction of the previous detection cycle. If the prediction result indicates that the rate of change of the clock deviation between the audio data input and output audio buffers is less than a certain threshold, it can be determined that the audio playback system is in a stable playback state during the previous detection cycle and a certain number of previous inspection cycles. This number is equal to the number of elements in the clock deviation set minus 1, and the specific number is not limited here.

[0049] In one embodiment, the step of calculating the first clock calibration amount based on the first clock deviation corresponding to the current detection period in the clock deviation set includes: matching the current clock calibration strategy corresponding to the first clock deviation in the current detection period from the preset clock calibration strategies, and obtaining the current clock calibration intensity factor corresponding to the current clock calibration strategy; calculating the clock smoothing calibration factor by the exponential moving average method, and obtaining the current clock calibration smoothing factor corresponding to the current detection period; and calculating the first clock calibration amount based on the preset minimum adjustable clock amount, the current clock calibration intensity factor, and the current clock calibration smoothing factor. First clock calibration value for:

[0050] in, This represents the current clock calibration smoothing factor corresponding to the current detection period. This represents the current clock calibration strength factor corresponding to the current clock calibration strategy. This indicates the preset minimum adjustable clock value.

[0051] In this embodiment, when calculating the first clock calibration amount, the current clock calibration intensity factor and the current clock calibration smoothing factor corresponding to the current detection cycle are combined. The current clock calibration intensity factor represents the correction strength. The greater the first clock deviation corresponding to the current detection cycle, the greater the corresponding current clock calibration intensity factor and the greater the correction strength. The current clock calibration smoothing factor represents the smoothness of the calibration. The larger the value, the greater the amount of clock adjustment in a single operation and the lower the smoothness of the calibration.

[0052] The current clock calibration intensity factor is determined according to the graded clock calibration strategy. Specifically, the clock deviation is pre-divided into multiple ranges, and each range corresponds to a preset clock calibration intensity factor. For the first clock deviation corresponding to the current detection cycle, the range to which the first clock deviation belongs is determined, and then the preset clock calibration intensity factor corresponding to the range is determined as the current clock calibration intensity factor corresponding to the unit clock calibration strategy.

[0053] For example, if | If |≤ 1ppm, a fine-tuning strategy will be adopted, corresponding to = 0.1; if 1ppm <| When the concentration is ≤5ppm, a mid-range adjustment strategy is adopted, corresponding to... = 0.3; if 5ppm <| If ≤ 10ppm, an emphasis strategy is adopted, corresponding to = 0.5; if | If the value is greater than 10 ppm, an emergency adjustment strategy will be adopted. = 0.8, the specific value is not specified here.

[0054] The minimum clock adjustability is a basic unit of quantization precision; for example, assuming... , , Then the first clock calibration amount Specific details are not specified here.

[0055] In one implementation, the current clock calibration smoothing factor corresponding to the current detection period for:

[0056] in, This represents the preset smoothing coefficient. It is the clock calibration smoothing factor corresponding to the previous detection cycle.

[0057] In this embodiment, the current clock calibration smoothing factor can be calculated based on the first clock deviation and smoothing coefficient corresponding to the current detection cycle. By combining the clock calibration smoothing factor of the previous detection cycle, a more accurate and smoothing factor can be obtained, thus improving the smoothness of clock calibration.

[0058] In one implementation, the step of predicting the clock drift trend of a set of clock deviations and determining a second clock calibration amount based on the prediction result includes: predicting the clock drift trend of a first clock deviation in the set of clock deviations using a linear regression algorithm to obtain a prediction result; wherein the prediction result is used to indicate the rate of change of the clock deviation between the audio data input and output audio buffers; and based on the prediction result, calculating a clock calibration amount to correct the clock drift trend within a specified prediction time window to obtain a second clock calibration amount. Second clock calibration value for:

[0059] Where T represents the specified forecast time window. represents the prediction result, and w represents the prediction weight coefficient.

[0060] Prediction results , represents the rate of change of the clock offset between the audio data input and output audio buffers. This rate of change is directional; a positive number indicates an increasing clock offset, and a negative number indicates a decreasing clock offset. The prediction time window T represents the number of detection cycles to be predicted, but the specific number is not limited here.

[0061] In one embodiment, the step of determining the sampling rate of the audio buffer for at least one future frame of audio data output, based on a first clock calibration value and a second clock calibration value, to obtain a target output sampling rate includes: superimposing the first clock calibration value and the second clock calibration value to obtain a target clock calibration value; decomposing the target clock calibration value into multiple unit calibration values ​​according to a specified correction step size, and determining the unit calibration value as the output sampling rate adjustment amount for at least one future frame; and calculating the sampling rate of the audio buffer for at least one future frame of audio data output, based on the output sampling rate adjustment amount, to obtain the target output sampling rate.

[0062] In this embodiment, the correction of the target clock calibration amount can be completed within a specified correction step size. The larger the correction step size, the smaller the clock amount calibrated each time. For example, assuming the target clock calibration amount is 0.654ppm and the correction step size is 15 steps, the clock amount adjusted in each step is 0.654 / 15 = 0.044ppm, which means that in the next 15 consecutive audio processing frames, the frequency adjustment will be increased by 0.044ppm per frame.

[0063] For the corresponding method embodiments described above, see [link to relevant documentation]. Figure 2 The diagram illustrates a device for calculating the audio output sampling rate. This device, applied to an audio playback system, includes: a detection module 20, used to detect the fill level of the audio buffer at a preset detection frequency during audio data playback; wherein the fill level indicates the current data volume of the audio buffer; an acquisition module 22, used to acquire first clock deviations corresponding to the most recent multiple detection cycles based on the fill level, obtaining a clock deviation set; wherein the first clock deviation indicates the ratio of clock deviation between audio data input and output to the audio buffer per unit time; a calculation module 24, used to calculate a first clock calibration amount based on the first clock deviation corresponding to the current detection cycle in the clock deviation set; a prediction module 26, used to predict the clock drift trend of the clock deviation set and determine a second clock calibration amount based on the prediction result; and a determination module 28, used to determine the sampling rate of at least one future frame of audio data output to the audio buffer based on the first clock calibration amount and the second clock calibration amount, obtaining a target output sampling rate.

[0064] The aforementioned audio output sampling rate calculation device periodically monitors the fill level of the audio buffer to estimate the instantaneous deviation of the clock. It calculates the calibration amount to compensate for the instantaneous deviation and the calibration amount to compensate for the long-term deviation, respectively, for both instantaneous and long-term clock deviations. Then, by adjusting the output sampling rate, the clock deviation is calibrated, which can fundamentally solve the problem of short-term and long-term clock domain mismatch, making clock calibration more real-time and accurate, and able to adapt to different deviation levels.

[0065] Optionally, the acquisition module 22 is specifically used for: calculating the difference between the fill level of the audio buffer detected in the current detection period and a preset baseline fill level to obtain the fill level deviation value corresponding to the current detection period; calculating the clock deviation ratio between the audio data input and output of the audio buffer within a unit time based on the fill level deviation value corresponding to the current detection period, the detection time interval of the preset detection frequency, and the current output sampling rate of the audio buffer to obtain the first clock deviation corresponding to the current detection period; acquiring the first clock deviations corresponding to multiple detection periods most recent to the current detection period in historical detection periods to obtain a clock deviation set; and the first clock deviation corresponding to the current detection period. for:

[0066] in, This indicates the fill level deviation value corresponding to the current testing cycle. I represents the current output sampling rate of the audio buffer, and I represents the detection time interval of the preset detection frequency.

[0067] Optionally, the baseline fill level is used to indicate the average fill level of the audio buffer over a certain period of time when the audio playback system is in a stable playback state.

[0068] Optionally, the calculation module 24 is specifically used for: matching the current clock calibration strategy corresponding to the first clock deviation in the current detection period from the preset clock calibration strategies, and obtaining the current clock calibration intensity factor corresponding to the current clock calibration strategy; calculating the clock smoothing calibration factor using the exponential moving average method to obtain the current clock calibration smoothing factor corresponding to the current detection period; calculating the first clock calibration amount based on the preset minimum clock adjustability, the current clock calibration intensity factor, and the current clock calibration smoothing factor; the first clock calibration amount for:

[0069] in, This represents the current clock calibration smoothing factor corresponding to the current detection period. This represents the current clock calibration strength factor corresponding to the current clock calibration strategy. This indicates the preset minimum adjustable clock value.

[0070] Optionally, the current clock calibration smoothing factor corresponding to the current detection period. for:

[0071] in, This represents the preset smoothing coefficient. It is the clock calibration smoothing factor corresponding to the previous detection cycle.

[0072] Optionally, the prediction module 26 is specifically used to: predict the clock drift trend of a first clock deviation in the clock deviation set using a linear regression algorithm, and obtain a prediction result; wherein the prediction result is used to indicate the rate of change of the clock deviation between the audio data input and output audio buffer; based on the prediction result, calculate a clock calibration amount to correct the clock drift trend within a specified prediction time window, and obtain a second clock calibration amount; the second clock calibration amount for:

[0073] Where T represents the specified forecast time window. The prediction result is represented by w, which represents the prediction weight coefficient.

[0074] Optionally, the determining module 28 is specifically used to: superimpose the first clock calibration amount and the second clock calibration amount to obtain a target clock calibration amount; decompose the target clock calibration amount into multiple unit calibration amounts according to a specified correction step size, and determine the unit calibration amount as the output sampling rate adjustment amount for at least one future frame; and calculate the sampling rate of the audio data output in the audio buffer for at least one future frame based on the output sampling rate adjustment amount to obtain the target output sampling rate.

[0075] This embodiment also provides an electronic device, including a processor and a memory. The memory stores machine-executable instructions that can be executed by the processor. The processor executes the machine-executable instructions to implement the above-described method for calculating the audio output sampling rate. This electronic device can be a server or a terminal device.

[0076] See Figure 3 As shown, the electronic device includes a processor 100 and a memory 101. The memory 101 stores machine-executable instructions that can be executed by the processor 100. The processor 100 executes the machine-executable instructions to implement the above-described method for calculating the audio output sampling rate.

[0077] Furthermore, Figure 3 The electronic device shown also includes a bus 102 and a communication interface 103, with the processor 100, the communication interface 103 and the memory 101 connected via the bus 102.

[0078] The memory 101 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 103 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc. The bus 102 may be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0079] The processor 100 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 100 or by instructions in software form. The processor 100 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this disclosure can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a readily available storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 101. The processor 100 reads information from memory 101 and, in conjunction with its hardware, completes the steps of the method described in the foregoing embodiments, for example: During the playback of audio data in the audio buffer, the fill level of the audio buffer is detected at a preset detection frequency; wherein, the fill level is used to indicate the current data volume of the audio buffer; based on the fill level, the first clock deviation corresponding to the most recent multiple detection cycles is obtained to obtain a clock deviation set; wherein, the first clock deviation is used to indicate the clock deviation ratio between the audio data input and output of the audio buffer per unit time; according to the first clock deviation corresponding to the current detection cycle in the clock deviation set, a first clock calibration amount is calculated; clock drift trend prediction is performed on the clock deviation set, and a second clock calibration amount is determined according to the prediction result; according to the first clock calibration amount and the second clock calibration amount, the sampling rate of at least one frame of audio data output to the audio buffer in the future is determined to obtain the target output sampling rate.

[0080] In this method, the instantaneous deviation of the clock is estimated by periodically monitoring the fill level of the audio buffer. For both instantaneous and long-term clock deviations, the calibration amount used to compensate for the instantaneous deviation and the calibration amount used to compensate for the long-term deviation are calculated respectively. Then, the clock deviation is calibrated by adjusting the output sampling rate. This method can fundamentally solve the problem of short-term and long-term clock domain mismatch, making clock calibration more real-time and accurate, and able to adapt to different deviation levels.

[0081] Optionally, the step of obtaining a set of clock deviations by acquiring the first clock deviations corresponding to the most recent multiple detection cycles based on the fill level includes: calculating the difference between the fill level of the audio buffer detected in the current detection cycle and a preset baseline fill level to obtain a fill level deviation value corresponding to the current detection cycle; calculating the ratio of clock deviations between the audio data input and output of the audio buffer per unit time based on the fill level deviation value corresponding to the current detection cycle, the detection time interval of the preset detection frequency, and the current output sampling rate of the audio buffer to obtain the first clock deviation corresponding to the current detection cycle; acquiring the first clock deviations corresponding to the most recent multiple detection cycles in historical detection cycles to obtain a set of clock deviations; and the first clock deviation corresponding to the current detection cycle. for:

[0082] in, This indicates the fill level deviation value corresponding to the current testing cycle. I represents the current output sampling rate of the audio buffer, and I represents the detection time interval of the preset detection frequency.

[0083] Optionally, the baseline fill level is used to indicate the average fill level of the audio buffer over a certain period of time when the audio playback system is in a stable playback state.

[0084] Optionally, the step of calculating the first clock calibration amount based on the first clock deviation corresponding to the current detection period in the clock deviation set includes: matching the current clock calibration strategy corresponding to the first clock deviation in the current detection period from a preset clock calibration strategy, and obtaining the current clock calibration intensity factor corresponding to the current clock calibration strategy; calculating the clock smoothing calibration factor using the exponential moving average method to obtain the current clock calibration smoothing factor corresponding to the current detection period; and calculating the first clock calibration amount based on a preset minimum adjustable clock amount, the current clock calibration intensity factor, and the current clock calibration smoothing factor; the first clock calibration amount... for:

[0085] in, This represents the current clock calibration smoothing factor corresponding to the current detection period. This represents the current clock calibration strength factor corresponding to the current clock calibration strategy. This indicates the preset minimum adjustable clock value.

[0086] Optionally, the current clock calibration smoothing factor corresponding to the current detection period. for:

[0087] in, This represents the preset smoothing coefficient. It is the clock calibration smoothing factor corresponding to the previous detection cycle.

[0088] Optionally, the step of predicting the clock drift trend of the clock deviation set and determining the second clock calibration amount based on the prediction result includes: predicting the clock drift trend of the first clock deviation in the clock deviation set using a linear regression algorithm to obtain a prediction result; wherein the prediction result is used to indicate the rate of change of the clock deviation between the audio data input and output audio buffer; based on the prediction result, calculating the clock calibration amount to correct the clock drift trend within a specified prediction time window to obtain the second clock calibration amount; the second clock calibration amount... for:

[0089] Where T represents the specified forecast time window. The prediction result is represented by w, which represents the prediction weight coefficient.

[0090] Optionally, the step of determining the sampling rate of the audio buffer for outputting at least one frame of audio data in the future, based on the first clock calibration value and the second clock calibration value, to obtain a target output sampling rate includes: superimposing the first clock calibration value and the second clock calibration value to obtain a target clock calibration value; decomposing the target clock calibration value into multiple unit calibration values ​​according to a specified correction step size, and determining the unit calibration value as the output sampling rate adjustment amount for at least one frame in the future; and calculating the sampling rate of the audio buffer for outputting at least one frame of audio data in the future, based on the output sampling rate adjustment amount, to obtain the target output sampling rate.

[0091] This embodiment also provides a computer-readable storage medium storing computer-executable instructions. When these computer-executable instructions are called and executed by a processor, they cause the processor to implement the aforementioned method for calculating the audio output sampling rate. For example: During the playback of audio data in the audio buffer, the fill level of the audio buffer is detected at a preset detection frequency; wherein, the fill level is used to indicate the current data volume of the audio buffer; based on the fill level, the first clock deviation corresponding to the most recent multiple detection cycles is obtained to obtain a clock deviation set; wherein, the first clock deviation is used to indicate the clock deviation ratio between the audio data input and output of the audio buffer per unit time; according to the first clock deviation corresponding to the current detection cycle in the clock deviation set, a first clock calibration amount is calculated; clock drift trend prediction is performed on the clock deviation set, and a second clock calibration amount is determined according to the prediction result; according to the first clock calibration amount and the second clock calibration amount, the sampling rate of at least one frame of audio data output to the audio buffer in the future is determined to obtain the target output sampling rate.

[0092] In this method, the instantaneous deviation of the clock is estimated by periodically monitoring the fill level of the audio buffer. For both instantaneous and long-term clock deviations, the calibration amount used to compensate for the instantaneous deviation and the calibration amount used to compensate for the long-term deviation are calculated respectively. Then, the clock deviation is calibrated by adjusting the output sampling rate. This method can fundamentally solve the problem of short-term and long-term clock domain mismatch, making clock calibration more real-time and accurate, and able to adapt to different deviation levels.

[0093] Optionally, the step of obtaining a set of clock deviations by acquiring the first clock deviations corresponding to the most recent multiple detection cycles based on the fill level includes: calculating the difference between the fill level of the audio buffer detected in the current detection cycle and a preset baseline fill level to obtain a fill level deviation value corresponding to the current detection cycle; calculating the ratio of clock deviations between the audio data input and output of the audio buffer per unit time based on the fill level deviation value corresponding to the current detection cycle, the detection time interval of the preset detection frequency, and the current output sampling rate of the audio buffer to obtain the first clock deviation corresponding to the current detection cycle; acquiring the first clock deviations corresponding to the most recent multiple detection cycles in historical detection cycles to obtain a set of clock deviations; and the first clock deviation corresponding to the current detection cycle. for:

[0094] in, This indicates the fill level deviation value corresponding to the current testing cycle. I represents the current output sampling rate of the audio buffer, and I represents the detection time interval of the preset detection frequency.

[0095] Optionally, the baseline fill level is used to indicate the average fill level of the audio buffer over a certain period of time when the audio playback system is in a stable playback state.

[0096] Optionally, the step of calculating the first clock calibration amount based on the first clock deviation corresponding to the current detection period in the clock deviation set includes: matching the current clock calibration strategy corresponding to the first clock deviation in the current detection period from a preset clock calibration strategy, and obtaining the current clock calibration intensity factor corresponding to the current clock calibration strategy; calculating the clock smoothing calibration factor using the exponential moving average method to obtain the current clock calibration smoothing factor corresponding to the current detection period; and calculating the first clock calibration amount based on a preset minimum adjustable clock amount, the current clock calibration intensity factor, and the current clock calibration smoothing factor; the first clock calibration amount... for:

[0097] in, This represents the current clock calibration smoothing factor corresponding to the current detection period. This represents the current clock calibration strength factor corresponding to the current clock calibration strategy. This indicates the preset minimum adjustable clock value.

[0098] Optionally, the current clock calibration smoothing factor corresponding to the current detection period. for:

[0099] in, This represents the preset smoothing coefficient. It is the clock calibration smoothing factor corresponding to the previous detection cycle.

[0100] Optionally, the step of predicting the clock drift trend of the clock deviation set and determining the second clock calibration amount based on the prediction result includes: predicting the clock drift trend of the first clock deviation in the clock deviation set using a linear regression algorithm to obtain a prediction result; wherein the prediction result is used to indicate the rate of change of the clock deviation between the audio data input and output audio buffer; based on the prediction result, calculating the clock calibration amount to correct the clock drift trend within a specified prediction time window to obtain the second clock calibration amount; the second clock calibration amount... for:

[0101] Where T represents the specified forecast time window. The prediction result is represented by w, which represents the prediction weight coefficient.

[0102] Optionally, the step of determining the sampling rate of the audio buffer for outputting at least one frame of audio data in the future, based on the first clock calibration value and the second clock calibration value, to obtain a target output sampling rate includes: superimposing the first clock calibration value and the second clock calibration value to obtain a target clock calibration value; decomposing the target clock calibration value into multiple unit calibration values ​​according to a specified correction step size, and determining the unit calibration value as the output sampling rate adjustment amount for at least one frame in the future; and calculating the sampling rate of the audio buffer for outputting at least one frame of audio data in the future, based on the output sampling rate adjustment amount, to obtain the target output sampling rate.

[0103] The computer program product for calculating the audio output sampling rate, including the method, apparatus, electronic device, and storage medium, provided in this disclosure includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the preceding method embodiments. For specific implementation details, please refer to the method embodiments, which will not be repeated here.

[0104] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system and apparatus described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0105] Furthermore, in the description of the embodiments of this disclosure, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this disclosure based on the specific circumstances.

[0106] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0107] In the description of this disclosure, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this disclosure and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this disclosure. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0108] Finally, it should be noted that the above embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A method for calculating the audio output sampling rate, applied to an audio playback system, characterized in that, The method includes: During the playback of audio data in the audio buffer, the fill level of the audio buffer is detected at a preset detection frequency; wherein, the fill level is used to indicate the current amount of data in the audio buffer; Based on the fill level, the first clock deviation corresponding to the most recent multiple detection cycles is obtained to obtain a clock deviation set; wherein, the first clock deviation is used to indicate the ratio of clock deviation between the audio data input and output audio buffer within a unit time. Calculate the first clock calibration amount based on the first clock deviation corresponding to the current detection cycle in the clock deviation set; The clock drift trend is predicted for the clock deviation set, and the second clock calibration amount is determined based on the prediction result; Based on the first clock calibration value and the second clock calibration value, the sampling rate of the audio buffer for outputting at least one frame of audio data in the future is determined, and the target output sampling rate is obtained.

2. The method according to claim 1, characterized in that, Based on the fill level, the step of obtaining the first clock offset corresponding to the most recent multiple detection cycles and obtaining the clock offset set includes: The difference between the fill level of the audio buffer detected in the current detection cycle and the preset baseline fill level is calculated to obtain the fill level deviation value corresponding to the current detection cycle. Based on the fill level deviation value corresponding to the current detection cycle, the detection time interval of the preset detection frequency, and the current output sampling rate of the audio buffer, calculate the clock deviation ratio between the audio data input and output of the audio buffer within a unit time to obtain the first clock deviation corresponding to the current detection cycle. Obtain the first clock deviation corresponding to the most recent detection period in the historical detection period to obtain the clock deviation set; The current detection cycle corresponds to the first clock deviation. for: in, This indicates the fill level deviation value corresponding to the current testing cycle. I represents the current output sampling rate of the audio buffer, and I represents the detection time interval of the preset detection frequency.

3. The method according to claim 2, characterized in that, The baseline fill level is used to indicate the average fill level of the audio buffer over a certain period of time when the audio playback system is in a stable playback state.

4. The method according to claim 2, characterized in that, The step of calculating the first clock calibration amount based on the first clock deviation corresponding to the current detection period in the clock deviation set includes: Match the current clock calibration strategy corresponding to the first clock deviation in the current detection period from the preset clock calibration strategy set, and obtain the current clock calibration intensity factor corresponding to the current clock calibration strategy. The clock smoothing calibration factor is calculated using the exponential moving average method to obtain the current clock calibration smoothing factor corresponding to the current detection cycle. The first clock calibration amount is calculated based on the preset minimum clock adjustability, the current clock calibration strength factor, and the current clock calibration smoothing factor. First clock calibration amount for: in, This represents the current clock calibration smoothing factor corresponding to the current detection period. This represents the current clock calibration strength factor corresponding to the current clock calibration strategy. This indicates the preset minimum adjustable clock value.

5. The method according to claim 4, characterized in that, Current clock calibration smoothing factor corresponding to the current detection period for: in, This represents the preset smoothing coefficient. It is the clock calibration smoothing factor corresponding to the previous detection cycle.

6. The method according to claim 1, characterized in that, The steps of predicting the clock drift trend of the clock deviation set and determining the second clock calibration amount based on the prediction result include: A linear regression algorithm is used to predict the clock drift trend of the first clock deviation in the clock deviation set, and a prediction result is obtained; wherein, the prediction result is used to indicate the rate of change of the clock deviation between the audio data input and output audio buffer; Based on the prediction results, the clock calibration amount to correct the clock drift trend within the specified prediction time window is calculated to obtain the second clock calibration amount. Second clock calibration amount for: Where T represents the specified forecast time window. The prediction result is represented by w, which represents the prediction weight coefficient.

7. The method according to claim 1, characterized in that, The step of determining the sampling rate of at least one future frame of audio data output to the audio buffer based on the first clock calibration value and the second clock calibration value, and obtaining the target output sampling rate, includes: The first clock calibration value and the second clock calibration value are superimposed to obtain the target clock calibration value; According to the specified correction step size, the target clock calibration amount is decomposed into multiple unit calibration amounts, and the unit calibration amount is determined as the output sampling rate adjustment amount for at least one future frame. Based on the output sampling rate adjustment, the sampling rate of the audio buffer for at least one future frame of audio data is calculated to obtain the target output sampling rate.

8. A device for calculating the audio output sampling rate, applied to an audio playback system, characterized in that, The device includes: The detection module is used to detect the fill level of the audio buffer at a preset detection frequency during the playback of audio data in the audio buffer; wherein the fill level is used to indicate the current amount of data in the audio buffer. The acquisition module is used to acquire the first clock deviation corresponding to the most recent multiple detection cycles based on the fill level, and obtain a clock deviation set; wherein, the first clock deviation is used to indicate the clock deviation ratio between the audio data input and output audio buffer in a unit time. The calculation module is used to calculate the first clock calibration amount based on the first clock deviation corresponding to the current detection cycle in the clock deviation set; The prediction module is used to predict the clock drift trend of the clock deviation set and determine the second clock calibration amount based on the prediction result. The determining module is used to determine the sampling rate of the audio buffer for outputting at least one frame of audio data in the future, based on the first clock calibration amount and the second clock calibration amount, so as to obtain the target output sampling rate.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, the processor executing the machine-executable instructions to implement the method for calculating the audio output sampling rate according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the method for calculating the audio output sampling rate as described in any one of claims 1-7.

Images