An audio synchronization control method, device, system, chip and medium

Abstract

An audio synchronization control method, apparatus, system, chip, and medium are disclosed. The method includes: in an audio playback state where audio data frames sent by an audio host are received, acquiring sampled data, i.e., the clock jitter of the starting frame of the audio data frame; determining the total clock jitter in the current audio playback state for each sampled data of the starting frame; obtaining a corresponding predicted value for the sampled data of the next starting frame based on the sampled data and filter coefficients corresponding to the current adaptive filter; updating the filter coefficients based on the prediction error corresponding to the predicted value; and determining the clock jitter prediction correction amount of the phase-locked loop (PLL) based on the prediction error and the total clock jitter, and adjusting the PLL clock. Through the learning of the adaptive filter during operation, the clock jitter of the audio host at the next moment can be tracked and predicted in real time, and the clock of the audio device can be dynamically adjusted, resulting in high accuracy of clock synchronization after correction and effectively avoiding audio transmission asynchrony.

Description

Technical Field

[0001] This application relates to the field of audio data transmission technology, and in particular to an audio synchronization control method, device, system, chip, and medium. Background Technology

[0002] An audio data transmission system includes an audio host and audio devices. For example, in a USB speaker, the USB host sends audio data, and the USB device receives data sent by the USB host. The data transmission uses a synchronous method. Because the USB host and USB device each have independent clocks generated by crystal oscillators, and crystal oscillators have errors, there is a certain deviation in the synchronization of the clocks at both ends. Over time, this can lead to data asynchrony between the USB host and USB device. For instance, the USB host may send data too quickly relative to the data being played by the USB device, causing the USB device to be unable to receive data in time, resulting in noise; conversely, the USB host may send data too slowly, causing the USB device to have no data to play, resulting in discontinuous audio playback. Summary of the Invention

[0003] To address at least one problem existing in the prior art, the purpose of this application is to provide an audio synchronization control method, device, system, chip, and medium. Through the learning of the adaptive filter during operation, it can track and predict the clock jitter of the audio host in real time, flexibly and effectively optimize the filtering performance, and dynamically adjust the clock of the audio device, so that the corrected clock synchronization accuracy is high, effectively avoiding the phenomenon of audio transmission asynchrony, and the audio quality is good and the algorithm is simple and easy to implement.

[0004] To achieve the above objectives, the audio synchronization control method provided in this application is applied to an audio device, and the method includes,

[0005] In the audio playback state of receiving audio data frames sent by the audio host, sampled data is acquired; wherein, the sampled data is the clock jitter of the starting frame of the audio data frame;

[0006] For each sampled data frame of the starting frame, determine the total clock jitter in the current audio playback state;

[0007] Based on the current sampling data and filter coefficients of the adaptive filter, the sampling data of the next starting frame is predicted to obtain the corresponding predicted value; based on the prediction error corresponding to the predicted value, the filter coefficients of the adaptive filter are updated.

[0008] Based on the prediction error and the total clock jitter, determine the clock jitter prediction correction amount for the phase-locked loop;

[0009] The clock of the phase-locked loop is adjusted based on the clock jitter prediction correction amount.

[0010] Furthermore, the method also includes,

[0011] The prediction error is determined based on the predicted value and the actual value of the next sampled data;

[0012] Based on the prediction error, a loss function is constructed;

[0013] Based on the loss function, an adaptive minimum mean square filter is used to process the minimum mean square error in order to determine new filter coefficients.

[0014] Furthermore, the method also includes,

[0015] The instantaneous gradient of the filter coefficients is determined by taking the derivative of the loss function; wherein the instantaneous gradient is determined based on the gradient step size, the prediction error, and the sampling data corresponding to the current adaptive filter.

[0016] The filter coefficients of the adaptive filter for the next starting frame are updated based on the instantaneous gradient and the current filter coefficients of the adaptive filter.

[0017] Furthermore, after the step of determining the instantaneous gradient of the filter coefficients, the method further includes,

[0018] Normalize the instantaneous gradient;

[0019] The normalized instantaneous gradient is determined based on the gradient step size, the prediction error, the sampled data corresponding to the current adaptive filter, and the square of the norm of the sampled data corresponding to the current adaptive filter.

[0020] Furthermore, after the step of normalizing the instantaneous gradient, the method further includes,

[0021] The normalized instantaneous gradient is then regularized;

[0022] The regularized instantaneous gradient is determined based on the regularization parameter, the gradient step size, the prediction error, the sampling data corresponding to the current adaptive filter, and the square of the norm.

[0023] Furthermore, the method also includes,

[0024] In response to receiving a playback start command sent by the audio host, the audio device enters the audio playback state;

[0025] The adaptive filter is initialized before acquiring the sampled data.

[0026] Further, the step of determining the clock jitter prediction correction amount of the phase-locked loop based on the prediction error and the total clock jitter includes,

[0027] The clock jitter prediction correction amount of the phase-locked loop is determined based on the predicted value of the sampled data with a first weight value and the total clock jitter with a second weight value; wherein the sum of the first weight value and the second weight value is a constant.

[0028] The prediction error is determined based on the predicted value and the actual value of the next sampled data;

[0029] In response to the prediction error being greater than or equal to a first threshold, the first weight value is reduced;

[0030] In response to the prediction error being less than or equal to the second threshold, the first weight value is increased;

[0031] In response to the prediction error being less than the first threshold and greater than the second threshold, the first weight value remains unchanged.

[0032] Furthermore, the method also includes,

[0033] Get the amount of data remaining in the buffer;

[0034] In response to the remaining data amount being greater than or equal to the third threshold, the clock jitter prediction correction amount of the phase-locked loop is increased;

[0035] In response to the remaining data amount being less than or equal to the fourth threshold, the clock jitter prediction correction amount of the phase-locked loop is reduced;

[0036] In response to the remaining data amount being less than the third threshold and greater than the fourth threshold, the clock jitter prediction correction amount of the phase-locked loop remains unchanged.

[0037] Furthermore, the method also includes,

[0038] If the total clock jitter exceeds the fifth threshold, a synchronization anomaly is determined to exist.

[0039] Furthermore, the method also includes,

[0040] Initialize the high-precision timer;

[0041] The sampling data is acquired using the high-precision timer;

[0042] The currently collected sampled data are added together to determine the current total clock jitter.

[0043] To achieve the above objectives, this application also provides an audio synchronization control device, applied to an audio device, the device comprising,

[0044] The sampling module is used to acquire sampling data in an audio playback state where an audio data frame sent by an audio host is received, and to determine the total clock jitter in the current audio playback state for each starting frame of the sampling data; wherein the sampling data is the clock jitter of the starting frame of the audio data frame.

[0045] The prediction module predicts the sampling data of the next starting frame based on the current sampling data and filter coefficients of the adaptive filter, and obtains the corresponding predicted value; based on the prediction error corresponding to the predicted value, it updates the filter coefficients of the adaptive filter.

[0046] The correction module determines the clock jitter prediction correction amount for the phase-locked loop based on the prediction error and the total clock jitter.

[0047] The adjustment module adjusts the clock of the phase-locked loop based on the clock jitter prediction correction amount.

[0048] To achieve the above objectives, this application also provides an audio synchronization control system, including:

[0049] An audio host is used to send audio data frames to audio devices.

[0050] An audio device is configured with an audio synchronization control device as described above; the audio device is used to receive the audio data frame and perform audio synchronization control through the audio synchronization control device.

[0051] To achieve the above objectives, this application also provides a chip including a memory and a processor; the memory stores computer instructions; the processor is configured to execute the instructions to perform the audio synchronization control method as described above.

[0052] To achieve the above objectives, this application provides a computer-readable storage medium storing computer instructions that, when executed, perform the steps of the audio synchronization control method described above.

[0053] This application discloses an audio synchronization control method, apparatus, system, chip, and medium. The method involves acquiring sampled data during audio playback while receiving audio data frames sent by an audio host; determining the total clock jitter in the current audio playback state for each starting frame's sampled data; predicting the sampled data for the next starting frame based on the current adaptive filter's sampled data and filter coefficients to obtain a corresponding predicted value; updating the adaptive filter's filter coefficients based on the prediction error corresponding to the predicted value; determining the phase-locked loop's clock jitter prediction correction amount based on the prediction error and the total clock jitter; and adjusting the phase-locked loop's clock according to the clock jitter prediction correction amount. Thus, through the adaptive filter's learning during operation, it can track and predict the clock jitter amount at the audio host in real time, flexibly and effectively optimizing filtering performance and dynamically adjusting the audio device's clock. This results in high accuracy of the corrected clock synchronization, effectively avoiding audio transmission asynchrony, and providing good audio quality with a simple and easy-to-implement algorithm.

[0054] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the application. Attached Figure Description

[0055] The accompanying drawings are provided to further illustrate the present application and form part of the specification. Together with the embodiments of the present application, they serve to explain the present application but do not constitute a limitation thereof. In the drawings:

[0056] Figure 1 This is a block diagram of the audio synchronization control system according to an embodiment of this application;

[0057] Figure 2 This is a flowchart of an audio synchronization control method according to an embodiment of this application;

[0058] Figure 3 A simplified diagram of the adaptive filter structure according to an embodiment of this application;

[0059] Figure 4 for Figure 3 A schematic diagram of the specific structure of the adaptive filter;

[0060] Figure 5 This is a flowchart of an audio synchronization control method according to another embodiment of this application;

[0061] Figure 6 This is a flowchart of an audio synchronization control method according to yet another embodiment of this application;

[0062] Figure 7 This is a flowchart of an audio synchronization control method according to another embodiment of this application. Detailed Implementation

[0063] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0064] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0065] It should be noted that the concepts of "first" and "second" mentioned in this application are only used to distinguish different devices, modules, units, data or locations, and are not used to limit the order or interdependence of the functions performed by these devices, modules, units, data or locations.

[0066] It should be noted that the terms "one" and "multiple" used in this application are illustrative rather than restrictive, and those skilled in the art should understand that, unless explicitly stated otherwise in the context, they should be understood as "one or more". "Multiple" should be understood as two or more.

[0067] First, it should be noted that for data asynchrony between audio devices and audio hosts, adaptive transmission or synchronous transmission can be used.

[0068] Adaptive transmission works by continuously adjusting the audio device's clock to ensure frequency synchronization between the USB host and the device. However, this constant clock adaptation means there is no continuous, accurate master clock within the device, leading to jitter in the audio stream.

[0069] In synchronous transmission, the audio device synchronizes audio data based on the start-of-frame (SOF) sent by the audio host. For example, for a full-speed USB device, the audio host sends the start-of-frame at an interval of 1.00ms ± 0.0005ms, while for a high-speed device, it sends it at an interval of 125μs ± 0.0625μs. However, in synchronous transmission, the audio device receives data as soon as the audio host sends it. But due to the error between the two clocks, prolonged operation can lead to a decrease in sound quality, such as distortion and noise. To address this, the audio device typically uses resampling, inserting or discarding samples in the buffer to ensure a stable data transmission rate. However, resampling inherently alters the audio data, easily leading to uneven data processing and audio quality degradation. Furthermore, this non-integer multiple resampling places very high demands on the algorithm, resulting in high implementation complexity and additional burden on the application processor.

[0070] Based on this, this application provides an audio synchronization control method, device, system, chip, and medium. Through the learning of the adaptive filter during operation, it can track and predict the clock jitter of the audio host in real time, flexibly and effectively optimize the filtering performance, and dynamically adjust the clock of the audio device, so that the corrected clock synchronization accuracy is high, effectively avoiding the phenomenon of audio transmission asynchrony, and the audio quality is good and the algorithm is simple and easy to implement.

[0071] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0072] Figure 1 This is a block diagram of an audio synchronization control system according to an embodiment of this application. Figure 1 As shown, the audio synchronization control system 10 includes an audio host 11 and an audio device 12.

[0073] The audio host 11 is used to send audio data frames to the audio device 12. The audio device 12 is configured with an audio synchronization control device 13. The audio device 12 is used to receive the audio data frames sent by the audio host 11 and to perform audio synchronization control through the audio synchronization control device 13.

[0074] In a specific example, the audio host 11 can be a USB audio host; the audio device 12 can be a USB audio device connected to the audio host 11 via USB.

[0075] Figure 2 This is a flowchart of an audio synchronization control method according to an embodiment of this application. This audio synchronization control method is applied to an audio device, and will be referred to below. Figure 2 The audio synchronization control method of this application is described in detail below.

[0076] In step 201, while in the audio playback state of receiving audio data frames sent by the audio host, sampling data is acquired.

[0077] The sampled data refers to the clock jitter of the starting frame of the audio data frame. Clock jitter refers to the amount by which the actual clock leads or lags behind the ideal clock edge.

[0078] Audio playback state refers to the operational state of receiving audio data frames from the audio host. Specifically, each time the play button on the audio host is pressed, it can be considered as entering an audio playback state, and each audio playback state is independent of the others. For example, in a specific example, an initialization is performed each time an audio playback state is entered.

[0079] In step 202, for each starting frame of sampled data, the total clock jitter in the current audio playback state is determined.

[0080] In this embodiment of the application, the method includes: initializing a high-precision timer; collecting sampling data through the high-precision timer; and adding the currently collected sampling data to determine the current total clock jitter.

[0081] In other words, a high-precision timer is used to record the clock jitter of the starting frame of each audio data frame, and the current clock jitter is added to the clock jitter of each previous starting frame in the current audio playback state to obtain the current total clock jitter sum_jitters.

[0082] In step 203, based on the current sampled data and filter coefficients of the adaptive filter, the sampled data of the next starting frame is predicted to obtain the corresponding predicted value; based on the prediction error corresponding to the predicted value, the filter coefficients of the adaptive filter are updated.

[0083] Specifically, such as Figure 3 and Figure 4 As shown, the adaptive filter can be an L-order LMS (Least Mean Square) adaptive filter. The current sampled data corresponding to the adaptive filter is the L most recent sampled data in L delay units (z^-1). The current adaptive filter coefficients are Specifically, W(n-1) can be... T ×X(n-1), that is, w1(n-1)x(n-1)+w2(n-1)x(n-2)+……+w L(n-1)x(nL) is used as the predicted value y(n) of the sampled data x(n) for the next starting frame. When the reference value dn is x(n), the prediction error e corresponding to this predicted value y(n) is... n For e n =x n -W(n-1) T ×X(n-1), based on this prediction error e n The filter coefficients of the adaptive filter are updated as follows:

[0084] In this embodiment of the application, the method further includes: determining the prediction error based on the predicted value and the actual value of the next sampled data; constructing a loss function based on the prediction error; and performing minimum mean square error processing using an adaptive minimum mean square filter based on the loss function to determine new filter coefficients. Specifically, the loss function can be constructed as follows: Loss(W) = E(e n 2 )=E{[x n -W(n-1) T ×X(n-1)] 2 That is, the loss function Loss(W) can be the expected value of the squared prediction error.

[0085] Furthermore, the method also includes: taking the derivative of the loss function to determine the instantaneous gradient of the filter coefficients; and updating the filter coefficients of the adaptive filter for the next starting frame based on the instantaneous gradient and the current filter coefficients of the adaptive filter.

[0086] The instantaneous gradient is determined based on the gradient step size, prediction error, and the sampling data corresponding to the current adaptive filter.

[0087] In a specific example, taking the derivative of the loss function Loss(W) yields the instantaneous gradient of the filter coefficients: G W =μe n ×X(n-1). Where μ is the gradient step size, e n For the prediction error, X(n-1) represents the sampled data corresponding to the current adaptive filter, i.e., the clock jitter of the starting frame of the current audio data frame. Furthermore, based on this instantaneous gradient G... W The filter coefficients of the adaptive filter can be updated, i.e., the updated filter coefficients are: W(n) = W(n-1) + μe n×X(n-1). Therefore, based on the minimum mean square error (MMSE), the instantaneous gradient descent adaptive algorithm can achieve learning prediction through the adaptive LMS filter. Compared with the Wiener filter, which requires matrix inversion, the complexity of this algorithm is greatly reduced, effectively reducing the burden on the application processor.

[0088] Furthermore, after determining the instantaneous gradient of the filter coefficients, the method also includes normalizing the instantaneous gradient.

[0089] The normalized instantaneous gradient is determined based on the gradient step size, prediction error, the sampled data corresponding to the current adaptive filter, and the square of the norm of the sampled data corresponding to the current adaptive filter.

[0090] Specifically, different gradient step sizes have a significant impact on the convergence speed of the filter coefficients. A larger gradient step size results in faster convergence, but may lead to fluctuations in the predicted coefficients, preventing accurate convergence to the optimal value. Conversely, a smaller gradient step size allows the filter coefficients to converge accurately to the optimal value, but convergence is slower. For standard LMS, the gradient step size *u* is greatly affected by variations in the magnitude of the sampled data and the magnitude of errors introduced by the data. Therefore, normalized LMS, or NLMS (Normalized Least Mean Squares), is typically used to make the gradient step size more stable, thus balancing the convergence speed and quality of the filter coefficients.

[0091] In a specific example, the normalized instantaneous gradient is: Where ||X(n-1)|| is the norm of the sampled data corresponding to the current adaptive filter. That is, in this case, the filter coefficients are updated as follows:

[0092] Furthermore, after normalizing the instantaneous gradient, the method also includes regularizing the normalized instantaneous gradient.

[0093] The regularized instantaneous gradient is determined based on the regularization parameter, gradient step size, prediction error, the sampled data corresponding to the current adaptive filter, and the square of the norm.

[0094] Specifically, by regularizing the normalized instantaneous gradient—that is, by introducing a regularization parameter s into the denominator of the instantaneous gradient—it is possible to effectively avoid ||X(n-1)|| 2The problem of excessively large instantaneous gradients caused by gradients equal to 0 or infinitely close to 0 is addressed by improving the stability of the gradient step size, allowing NLMS to operate in a stable state, which helps to further optimize the convergence speed and convergence quality of the filter coefficients.

[0095] In a specific example, the instantaneous gradient after regularization is: That is, in this case, the filter coefficients are updated as follows:

[0096] In step 204, the clock jitter prediction correction amount of the phase-locked loop is determined based on the prediction error and the total clock jitter.

[0097] Specifically, corresponding weights can be assigned to the prediction error and the total clock jitter to determine the clock jitter prediction correction amount of the phase-locked loop. It is understood that this application does not impose specific restrictions on the configuration method of these weights.

[0098] In the embodiments of this application, such as Figure 5 As shown, step 204 may include the following sub-steps:

[0099] Step 2041: Determine the clock jitter prediction correction amount of the phase-locked loop based on the predicted value of the sampled data with the first weight value and the total clock jitter with the second weight value.

[0100] The sum of the first weight value and the second weight value is a constant, which can specifically be 1.

[0101] Step 2042: Determine the prediction error based on the predicted value and the actual value of the next sampled data.

[0102] Step 2043: In response to the prediction error being greater than or equal to the first threshold, decrease the first weight value.

[0103] Step 2044: In response to the prediction error being less than or equal to the second threshold, increase the first weight value.

[0104] Step 2045: In response to the prediction error being less than the first threshold and greater than the second threshold, the first weight value remains unchanged.

[0105] Specifically, the prediction error e n The magnitude of indicates the confidence level of the predicted value y(n). That is, the prediction error e n The larger the value of the first weight, the lower the reliability of the predicted value y(n), indicating a more accurate prediction. The smaller the difference between the predicted value y(n) and the true value x(n), the greater the first weight and the smaller the second weight. Conversely, the larger the prediction error e, the lower the reliability of the predicted value y(n) and the smaller the difference between the predicted value y(n) and the true value x(n). nThe smaller the value, the greater the reliability of the predicted value y(n), indicating that the prediction is less accurate and the difference between the predicted value y(n) and the true value x(n) is greater. This results in poor NLMS convergence. Therefore, the first weight value should be reduced and the second weight value should be increased.

[0106] In the embodiments of this application, such as Figure 6 As shown, the method may further include the following steps:

[0107] Step 2046: Obtain the remaining amount of data in the buffer.

[0108] Step 2047: In response to the remaining data amount being greater than or equal to the third threshold, increase the clock jitter prediction correction amount of the phase-locked loop.

[0109] Step 2048: In response to the remaining data amount being less than or equal to the fourth threshold, reduce the clock jitter prediction correction amount of the phase-locked loop.

[0110] Step 2049: In response to the remaining data amount being less than the third threshold and greater than the fourth threshold, the clock jitter prediction correction amount of the phase-locked loop is kept unchanged.

[0111] In other words, the PLL clock can be further corrected based on the remaining data in the buffer. Specifically, when the remaining data in the buffer is greater than or equal to the third threshold (i.e., too much data in the buffer and the PLL clock of the audio device is too slow), the clock jitter prediction correction of the PLL is increased. When the remaining data in the buffer is less than or equal to the fourth threshold (i.e., too little data in the buffer and the PLL clock of the audio device is too fast), the clock jitter prediction correction of the PLL is decreased. When the remaining data in the buffer is less than the third threshold but greater than the fourth threshold (i.e., the remaining data in the buffer is stable and the synchronization between the host clock and the device clock is good), the clock jitter prediction correction of the PLL remains unchanged.

[0112] In step 205, the clock of the phase-locked loop is adjusted according to the clock jitter prediction correction amount.

[0113] Specifically, based on the aforementioned clock jitter prediction correction amount, the phase-locked loop clock can be adjusted, and the relevant driving functions of the phase-locked loop clock can be called to adjust the frequency so that the audio device can achieve accurate synchronization with the starting frame clock of the next moment.

[0114] It should be noted that the phase-locked loop (PLL) of the audio device is connected to the digital audio interface (DAI). Since the clock of the DAI originates from the PLL, the clock of the DAI can be adjusted by correcting the clock of the PLL. Furthermore, this PLL supports step-by-step real-time online debugging, preferably using a small step size.

[0115] In this embodiment of the application, the method further includes: determining a synchronization anomaly in response to the total clock jitter exceeding a fifth threshold. That is, when the total clock jitter is too large, for example, greater than two audio sampling periods, a synchronization anomaly is determined to exist. In this case, to avoid triggering hardware failure by using the above-mentioned audio synchronization control method, a corresponding abnormal synchronization control strategy under the synchronization anomaly state can be activated to prevent triggering hardware failure.

[0116] In this embodiment of the application, the method further includes: in response to receiving a playback start command sent by the audio host, the audio device enters the audio playback state; and before acquiring the sampling data, the adaptive filter is initialized.

[0117] The present application will be further explained and illustrated below through a specific embodiment.

[0118] Figure 7 The flowchart of the audio synchronization control method according to this specific embodiment is shown below. Figure 7 As shown, the audio synchronization control method includes the following steps:

[0119] Step 301: Initialize the adaptive filter.

[0120] This includes the initialization of filter coefficients, sampled data, gradient step size, and regularization parameters.

[0121] Step 302: Based on the current sampling data and filter coefficients of the adaptive filter, predict the sampling data of the next starting frame to obtain the corresponding predicted value.

[0122] Step 303: Calculate the prediction error corresponding to the predicted value based on the predicted value and the sampling data of the next starting frame.

[0123] Step 304: Based on the prediction error and the current sampled data, determine the regularized and normalized instantaneous gradient.

[0124] Step 305: Update the filter coefficients of the adaptive filter based on the instantaneous gradient, and then return to step 302 until exiting the current audio playback state.

[0125] In a specific example, after the audio playback state is completed, the predicted value sequence y(n) and the prediction error sequence e(n) can be output.

[0126] In summary, the audio synchronization control method according to the embodiments of this application acquires sampled data during audio playback while receiving audio data frames sent by the audio host; determines the total clock jitter in the current audio playback state for each starting frame's sampled data; predicts the sampled data of the next starting frame based on the current adaptive filter's corresponding sampled data and filter coefficients to obtain a corresponding predicted value; updates the adaptive filter's filter coefficients based on the prediction error corresponding to the predicted value; determines the phase-locked loop's clock jitter prediction correction amount based on the prediction error and the total clock jitter; and adjusts the phase-locked loop's clock according to the clock jitter prediction correction amount. Therefore, through the adaptive filter's learning during operation, it can track and predict the clock jitter amount at the audio host in real time, flexibly and effectively optimizing the filtering performance and dynamically adjusting the audio device's clock. This results in high accuracy of the corrected clock synchronization, effectively avoiding audio transmission asynchrony, and providing good audio quality with a simple and easy-to-implement algorithm.

[0127] like Figure 1 As shown, the audio synchronization control device 13 provided in this application embodiment includes a sampling module 131, a prediction module 132, a correction module 133, and an adjustment module 134.

[0128] The sampling module 131 is used to acquire sampling data in the audio playback state when receiving audio data frames sent by the audio host, and to determine the total clock jitter in the current audio playback state for each starting frame of the sampling data; wherein the sampling data is the clock jitter of the starting frame of the audio data frame.

[0129] The prediction module 132 predicts the sampling data of the next starting frame based on the current sampling data and filtering coefficients of the adaptive filter, obtains the corresponding predicted value, and updates the filtering coefficients of the adaptive filter based on the prediction error corresponding to the predicted value.

[0130] The correction module 133 determines the clock jitter prediction correction amount of the phase-locked loop based on the prediction error and the total clock jitter.

[0131] The adjustment module 134 adjusts the clock of the phase-locked loop based on the clock jitter prediction correction amount.

[0132] In this embodiment of the application, the prediction module 132 is further configured to: determine the prediction error based on the predicted value and the actual value of the next sampled data; construct a loss function based on the prediction error; and perform minimum mean square error processing using an adaptive minimum mean square filter based on the loss function to determine new filter coefficients.

[0133] Furthermore, the prediction module 132 is also used to: differentiate the loss function to determine the instantaneous gradient of the filter coefficients; wherein the instantaneous gradient is determined based on the gradient step size, the prediction error and the sampling data corresponding to the current adaptive filter; and update the filter coefficients of the adaptive filter for the next starting frame according to the instantaneous gradient and the filter coefficients of the current adaptive filter.

[0134] Furthermore, the prediction module 132 is also used to: normalize the instantaneous gradient after determining the instantaneous gradient of the filter coefficients. The normalized instantaneous gradient is determined based on the gradient step size, the prediction error, the sampled data corresponding to the current adaptive filter, and the square of the norm of the sampled data corresponding to the current adaptive filter.

[0135] Furthermore, the prediction module 132 is also used to: normalize the normalized instantaneous gradient after normalizing it. The normalized instantaneous gradient is determined based on the regularization parameter, gradient step size, prediction error, the sampled data corresponding to the current adaptive filter, and the square of the norm.

[0136] In this embodiment of the application, the audio synchronization control device 13 further includes an initialization module (not shown in the figure). The initialization module responds to the playback start command sent by the audio host, causes the audio device to enter the audio playback state, and initializes the adaptive filter before acquiring the sampling data.

[0137] In this embodiment, the correction module 133 is specifically configured to: determine the clock jitter prediction correction amount of the phase-locked loop based on the predicted value of the sampled data with a first weight value and the total clock jitter with a second weight value; determine the prediction error based on the predicted value and the actual value of the next sampled data; decrease the first weight value in response to a prediction error greater than or equal to a first threshold; increase the first weight value in response to a prediction error less than or equal to a second threshold; and keep the first weight value unchanged in response to a prediction error less than the first threshold and greater than the second threshold. The sum of the first weight value and the second weight value is a constant.

[0138] In this embodiment of the application, the correction module 133 is further configured to: obtain the remaining amount of data in the buffer; increase the clock jitter prediction correction amount of the phase-locked loop in response to the remaining amount of data being greater than or equal to a third threshold; decrease the clock jitter prediction correction amount of the phase-locked loop in response to the remaining amount of data being less than or equal to a fourth threshold; and keep the clock jitter prediction correction amount of the phase-locked loop unchanged in response to the remaining amount of data being less than the third threshold and greater than the fourth threshold.

[0139] In this embodiment of the application, the sampling module 131 is further configured to: determine that there is a synchronization abnormality in response to the total clock jitter being greater than a fifth threshold.

[0140] In this embodiment of the application, the sampling module 131 is further used to: initialize a high-precision timer; collect sampling data through the high-precision timer; and add the currently collected sampling data to determine the current total clock jitter.

[0141] It should be noted that the explanation of the audio synchronization control method in the above embodiments also applies to the audio synchronization control device in the above embodiments, and will not be repeated here.

[0142] In one embodiment of this application, an audio synchronization control system is also provided. For example... Figure 1 As shown, the audio synchronization control system 10 includes an audio host 11 and an audio device 12.

[0143] The audio host 11 is used to send audio data frames to the audio device 12. The audio device 12 is configured with the audio synchronization control device 13 in the above embodiment. The audio device 12 is used to receive audio data frames and perform audio synchronization control through the audio synchronization control device 13.

[0144] In one embodiment of this application, a chip is also provided, which includes a memory and a processor. The memory stores computer instructions; the processor is configured to execute the instructions to perform the audio synchronization control method in the above embodiments.

[0145] In one embodiment of this application, a computer-readable storage medium is also provided. This computer-readable storage medium may be included in the system described in the above embodiments, or it may exist independently and not assembled into the system. The computer-readable storage medium carries one or more computer instructions, which, when executed, implement the steps of the audio synchronization control method of the above embodiments.

[0146] In embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0147] It should be understood that although the steps in the flowcharts in the accompanying drawings are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0148] It will be understood by those skilled in the art that the above are merely preferred embodiments of this application and are not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An audio synchronization control method, characterized by, Applied to an audio device, the method includes, In the audio playback state of receiving audio data frames sent by the audio host, sampled data is acquired; wherein, the sampled data is the clock jitter of the starting frame of the audio data frame; For each sampled data frame of the starting frame, determine the total clock jitter in the current audio playback state; Based on the current sampling data and filter coefficients of the adaptive filter, the sampling data of the next starting frame is predicted to obtain the corresponding predicted value; based on the prediction error corresponding to the predicted value, the filter coefficients of the adaptive filter are updated. Based on the prediction error and the total clock jitter, determine the clock jitter prediction correction amount for the phase-locked loop; The clock of the phase-locked loop is adjusted based on the clock jitter prediction correction amount.

2. The audio synchronization control method of claim 1, wherein, The method also includes, The prediction error is determined based on the predicted value and the actual value of the next sampled data; Based on the prediction error, a loss function is constructed; Based on the loss function, an adaptive minimum mean square filter is used to process the minimum mean square error in order to determine new filter coefficients.

3. The audio synchronization control method of claim 2, wherein, The method also includes, The instantaneous gradient of the filter coefficients is determined by taking the derivative of the loss function; wherein the instantaneous gradient is determined based on the gradient step size, the prediction error, and the sampling data corresponding to the current adaptive filter. The filter coefficients of the adaptive filter for the next starting frame are updated based on the instantaneous gradient and the current filter coefficients of the adaptive filter.

4. The audio synchronization control method of claim 3, wherein, After the step of determining the instantaneous gradient of the filter coefficients, the method further includes, Normalize the instantaneous gradient; The normalized instantaneous gradient is determined based on the gradient step size, the prediction error, the sampled data corresponding to the current adaptive filter, and the square of the norm of the sampled data corresponding to the current adaptive filter.

5. The audio synchronization control method of claim 4, wherein, After the step of normalizing the instantaneous gradient, the method further includes, The normalized instantaneous gradient is then regularized; The regularized instantaneous gradient is determined based on the regularization parameter, the gradient step size, the prediction error, the sampling data corresponding to the current adaptive filter, and the square of the norm.

6. The audio synchronization control method according to claim 1, characterized in that, The method also includes, In response to receiving a playback start command sent by the audio host, the audio device enters the audio playback state; The adaptive filter is initialized before acquiring the sampled data.

7. The audio synchronization control method according to claim 1, characterized in that, The step of determining the clock jitter prediction correction amount for the phase-locked loop based on the prediction error and the total clock jitter is... include, The clock jitter prediction correction amount of the phase-locked loop is determined based on the predicted value of the sampled data with a first weight value and the total clock jitter with a second weight value; wherein the sum of the first weight value and the second weight value is a constant. The prediction error is determined based on the predicted value and the actual value of the next sampled data; In response to the prediction error being greater than or equal to a first threshold, the first weight value is reduced; In response to the prediction error being less than or equal to the second threshold, the first weight value is increased; In response to the prediction error being less than the first threshold and greater than the second threshold, the first weight value remains unchanged.

8. The audio synchronization control method according to claim 1, characterized in that, The method also includes, Get the amount of data remaining in the buffer; In response to the remaining data amount being greater than or equal to the third threshold, the clock jitter prediction correction amount of the phase-locked loop is increased; In response to the remaining data amount being less than or equal to the fourth threshold, the clock jitter prediction correction amount of the phase-locked loop is reduced; In response to the remaining data amount being less than the third threshold and greater than the fourth threshold, the clock jitter prediction correction amount of the phase-locked loop remains unchanged.

9. The audio synchronization control method according to claim 1, characterized in that, The method also includes, If the total clock jitter exceeds the fifth threshold, a synchronization anomaly is determined to exist.

10. The audio synchronization control method according to claim 1, characterized in that, The method also includes, Initialize the high-precision timer; The sampling data is acquired using the high-precision timer; The currently collected sampled data are added together to determine the current total clock jitter.

11. An audio synchronization control device, characterized in that, Applied to audio devices, the device includes, The sampling module is used to acquire sampling data in an audio playback state where an audio data frame sent by an audio host is received, and to determine the total clock jitter in the current audio playback state for each starting frame of the sampling data; wherein the sampling data is the clock jitter of the starting frame of the audio data frame. The prediction module predicts the sampling data of the next starting frame based on the current sampling data and filter coefficients of the adaptive filter, obtains the corresponding prediction value, and updates the filter coefficients of the adaptive filter based on the prediction error corresponding to the prediction value. The correction module determines the clock jitter prediction correction amount for the phase-locked loop based on the prediction error and the total clock jitter. The adjustment module adjusts the clock of the phase-locked loop based on the clock jitter prediction correction amount.

12. An audio synchronization control system, characterized in that, include, An audio host is used to send audio data frames to audio devices. An audio device is configured with the audio synchronization control device as described in claim 11; the audio device is used to receive the audio data frame and perform audio synchronization control through the audio synchronization control device.

13. A chip, characterized in that, It includes a memory and a processor; the memory stores computer instructions; the processor is configured to execute the instructions to perform the audio synchronization control method according to any one of claims 1-10.

14. A computer-readable storage medium, characterized in that, It stores computer instructions, which, when executed, perform the steps of the audio synchronization control method according to any one of claims 1 to 10.

Images