Method, apparatus, and system for isochronous audio playback synchronization

Abstract

Systems, methods, apparatuses, programs, and storage media for synchronizing two or more audio channel signals processed by separate audio playback chains are described. The method is performed for each of two or more audio channel signals. The method includes receiving a delay signal for a respective audio playback chain. Based on the delay signal, a clock rate is estimated using a first DLL. Based on the clock rate, a sampling rate is estimated using a PLL. Based on the sampling rate, the respective audio channel signal is resampled to generate a resampled audio channel signal. The resampled audio channel signal is delayed based on the delay signal to generate a delayed resampled audio channel signal such that the two or more delayed resampled audio channel signals remain time-synchronized when played back.

Description

Cross-references to related applications

[0001] This application claims priority to U.S. Provisional Application Serial No. 63 / 637,183, filed April 22, 2024, and U.S. Provisional Application Serial No. 63 / 787,188, filed April 11, 2025, each of which is incorporated herein by reference in its entirety. Technical Field

[0002] This disclosure relates to audio synchronization, and more specifically to methods and apparatus for synchronizing audio channels processed by different audio playback chains. Background Technology

[0003] Synchronizing audio channels processed by different playback chains is a challenging problem in audio engineering. Playback chains typically consist of a series of components, including digital signal processing (DSP) units, digital-to-analog converters (DACs), amplifiers, and speakers, each introducing its own processing time and potential variability. When multiple playback chains operate simultaneously, even small time differences can lead to phase mismatch, comb filtering, unwanted echoes, or spatial imaging failures in multi-channel audio systems. These problems are particularly pronounced in environments such as surround sound setups, live sound reinforcement, multi-room audio systems, and network audio applications.

[0004] Several factors can lead to synchronization problems. Hardware differences (such as variations in DAC processing speed or amplifier latency) can cause timing mismatches between channels. In digital audio systems, differences in sampling rate clocks between devices can cause drift, leading to a gradual loss of synchronization over time. In networked audio environments, jitter and buffering delays caused by variations in data transmission speeds can further exacerbate this problem, making real-time synchronization difficult. Even in offline scenarios, software-based processing (such as real-time DSP adjustments, equalization, and dynamic range compression) can introduce variable latency between different playback chains.

[0005] Therefore, a synchronization technology is needed to enable synchronized playback of audio channels processed by different audio playback chains. Summary of the Invention

[0006] In view of the foregoing, this disclosure provides methods, apparatus, and programs for synchronizing two or more audio channel signals processed by separate audio playback chains and for estimating clock rates based on delay signals, as well as computer-readable storage media having the features of the respective independent claims.

[0007] According to a first aspect of this disclosure, a method for synchronizing two or more audio channel signals, each audio channel signal being processed by a separate audio playback chain, is provided. The method can be performed for each audio channel signal. A delayed signal of the corresponding audio playback chain can be received. A clock rate can be estimated / determined using a delay-locked loop (DLL) based on the delayed signal. The clock rate can be estimated such that the delayed signal remains substantially constant over time when the clock rate is applied to the audio channel signal. The clock rate can correspond to or be related to the conversion rate of a digital-to-analog converter (DAC) of the corresponding audio playback chain. A sampling rate can be estimated / determined using a phase-locked loop (PLL) based on the clock rate. The audio channel signals can be resampled based on the sampling rate to generate resampled audio channel signals. The audio channel signals can be delayed based on the delayed signal to generate delayed resampled audio channel signals, such that the two or more delayed resampled audio channel signals remain time-synchronized during playback.

[0008] By employing this synchronization method, two or more audio channel signals can be synchronized even if the corresponding audio playback chain introduces completely different delays to the corresponding audio channel signals.

[0009] In some embodiments, the method may further include applying a clock rate to the delayed resampled audio channel signal to set the rate at which samples of the delayed resampled audio channel signal are transmitted through the corresponding audio playback chain.

[0010] In some embodiments, setting the rate at which samples of the resampled audio channel signal with transmission delay through the corresponding audio playback chain are transmitted may include providing samples at the set rate to the audio interface (i.e., an interface for receiving digital audio signals) of the corresponding audio playback chain. This audio interface may, for example, be the host of a Bluetooth initiating device or a digital-to-analog converter (DAC).

[0011] In some embodiments, estimating the clock rate using a DLL based on a delayed signal may include: estimating a first clock rate using a first DLL level of the DLL based on a first error signal between the delayed signal and a reference delay. This first clock rate may be estimated such that the delayed signal remains substantially constant over time when the first clock rate is applied to the audio channel signal. Further, pairs of the first clock rate and the first error signal may be accumulated over time until a threshold is reached (e.g., a threshold related to the number of pairs of the first clock rate and the first error signal). After reaching the threshold, a second clock rate may be estimated based on the accumulated pairs. The second clock rate may correspond to the case where the first error signal is equal to zero. Further, delay signal processing may switch from a first DLL level of the DLL to a second DLL level of the DLL (after the second clock rate is estimated). Then, a third clock rate may be estimated / determined using the second DLL level based on the delayed signal and the second clock rate. This third clock rate may be estimated such that the delayed signal remains substantially constant over time when the third clock rate is applied to the signal fed to the audio channel signal. The output clock rate may correspond to the first clock rate before switching from the first DLL level to the second DLL level, and may correspond to the third clock rate after switching from the first DLL level to the second DLL level.

[0012] By using a two-level DLL, a stable clock rate can be estimated even if the input delay signal quantization is coarse (e.g., due to inaccurate timers in the audio playback chain).

[0013] In some embodiments, each audio channel signal may include multiple audio sub-channel signals processed by the same audio playback chain, i.e., multiple audio channel signals may use the same audio playback chain or substantially the same audio playback chain, for example, multiple local audio channel signals.

[0014] In some embodiments, each audio playback chain can be either a local audio playback chain or a remote audio playback chain. A remote audio playback chain can include either a Bluetooth Low Energy audio playback chain or a WiFi audio playback chain.

[0015] In some embodiments, the delay signal of a corresponding audio playback chain can indicate audio playback delay, which includes the local presentation delay of the audio signal if the corresponding audio playback chain is a local audio playback chain; or, if the corresponding audio playback chain is a remote audio playback chain, the audio playback delay includes the transmission delay of the audio signal over the wireless medium and the remote presentation delay of the audio signal. Local presentation delay can be the delay from when the audio signal enters the DAC to when it is played back by the speaker. Remote presentation delay can be the delay from when the audio signal is received by the remote portion of the remote audio playback chain to when it is played back by the speaker.

[0016] In some embodiments, estimating the sampling rate using a PLL based on the clock rate may include smoothing the clock rate to estimate / determine the sampling rate.

[0017] In some embodiments, resampling the corresponding audio channel signal based on the sampling rate can convert the source digital audio sampling rate of the audio channel signal into a sampling rate. This source digital audio sampling rate can be a sampling rate associated with the codec used in the audio playback chain.

[0018] In some embodiments, generating delayed resampled audio channel signals such that two or more delayed resampled audio channel signals remain time-synchronized during playback may include delaying the resampled audio channel signals according to the difference between a preset delay and a delayed signal, wherein the preset delay is equal for all audio channel signals.

[0019] According to a second aspect of this disclosure, a two-stage DLL-based method for estimating clock rates based on delayed signals is provided. A delayed signal can be received. This delayed signal can correspond to a signal delay on a signal transmission chain (i.e., a signal transmission chain including digital processing). A first clock rate can be estimated using a first DLL stage based on a first error signal between the delayed signal and a reference delay. This first clock rate can be estimated such that when the first clock rate is applied to a signal fed to the signal transmission chain, the delayed signal remains substantially constant over time. Further, pairs of the first clock rate and the first error signal can be accumulated over time until a threshold is reached (e.g., a threshold related to the number of pairs of the first clock rate and the first error signal). After reaching the threshold, a second clock rate can be estimated based on the accumulated pairs. The second clock rate can correspond to the case where the first error signal is equal to zero. Further, delayed signal processing can be switched from a first DLL stage to a second DLL stage (after the second clock rate is estimated). Then, a third clock rate can be estimated / determined using the second DLL stage based on the delayed signal and the second clock rate. This third clock rate can be estimated such that when the third clock rate is applied to a signal fed to the signal transmission chain, the delayed signal remains substantially constant over time. The output clock rate can correspond to the first clock rate before switching from the first DLL level to the second DLL level, and can correspond to the third clock rate after switching from the first DLL level to the second DLL level.

[0020] By using this two-stage DLL method, the first clock rate can be quickly estimated, and after estimating a certain number of first clock rate samples, a stable third clock rate can be estimated. Therefore, even if the delay signal input to the two-stage DLL changes significantly over time (e.g., due to coarse quantization), a stable clock rate can be achieved.

[0021] In some embodiments, estimating the first clock rate using the first DLL level may include: applying a first averaging function to the delayed signal to generate a first averaged delay signal. Further, a first error signal may be determined between the first averaged delay signal and a reference delay. The reference delay may be a delay acquired during signal transmission chain startup. The first clock rate may be estimated by a proportional-derivative controller based on the error signal. The clock rate estimation by the proportional-derivative controller may further be based on the reference clock rate.

[0022] In some embodiments, estimating the second clock rate corresponding to a first error signal equal to zero based on accumulated pairs may include generating a histogram of pairs of the first clock rate and the first error signal. Further, the second clock rate may be estimated based on the N most frequent pairs of the first clock rate and the first error signal in the histogram, where N is a positive integer. Estimating the second clock rate based on the N most frequent pairs of the first clock rate and the first error signal in the histogram can be performed using linear regression; that is, a best-fit linear function can be found for the N most frequent pairs using linear regression, and this linear function can be used to determine the clock rate corresponding to a zero error signal.

[0023] In some embodiments, estimating the third clock rate using the second DLL level may include: applying a second averaging function to the delayed signal to generate a second averaged delayed signal. Further, a second error signal may be determined between the second averaged delayed signal and a delayed version of the second averaged delayed signal. The third clock rate may be estimated / determined by a proportional controller based on the second error signal and the second clock rate.

[0024] In some embodiments, the average window of the first averaging function may be shorter than the average window of the second averaging function. For example, the length of the average window of the first averaging function may be equal to 1 second, and the length of the average window of the second averaging function may be equal to 1 minute.

[0025] Any embodiment according to the second aspect can be combined with a two-level DLL embodiment according to the first aspect.

[0026] The first and second aspects of this disclosure can be implemented via an apparatus. The apparatus may include a processor and a memory coupled to the processor. The processor may be adapted to perform the methods described according to various aspects and embodiments of this disclosure.

[0027] The various aspects of this disclosure can be implemented via a program. When the instructions of the program are executed by a processor, the processor can perform the various aspects and embodiments of this disclosure. A computer-readable storage medium can store the program. Such a computer-readable storage medium may include memory devices, such as those described herein, including but not limited to random access memory (RAM) devices, read-only memory (ROM) devices, etc. Therefore, some of the innovative aspects of the subject matter described in this disclosure can be implemented via one or more computer-readable storage media on which software is stored.

[0028] It will be appreciated that the apparatus features and method steps can be interchanged in various ways. In particular, as those skilled in the art will recognize, the details of the disclosed methods can be implemented by the corresponding apparatus (or system) and vice versa. Furthermore, any statement made above regarding the method is to be understood to equally apply to the corresponding apparatus (or system) and vice versa. Attached Figure Description

[0029] The following explanation of exemplary embodiments of this disclosure is based on the accompanying drawings, in which: Figure 1 The diagram schematically illustrates two different audio playback chains used for two audio channel signals. Figure 2 The illustration schematically depicts Bluetooth LE Audio processing latency. Figure 3 The timing diagram illustrates the bus timings for Bluetooth LE audio. Figure 4 An example of two synchronization subsystems for two audio channel signals according to an embodiment of the present disclosure is illustrated schematically. Figure 5 An example of a two-level delay-locked loop DLL according to an embodiment of the present disclosure is schematically illustrated. Figure 6 An example implementation of a two-level DLL according to embodiments of the present disclosure is illustrated schematically. Figure 7 An example histogram of error and clock rate samples, and an estimate of the clock rate, are schematically illustrated according to embodiments of the present disclosure. Figure 8 This is a flowchart illustrating an example of a method for synchronizing audio channel signals processed by different audio playback chains according to an embodiment of the present disclosure. Figure 9 This is a flowchart illustrating an example of a method for estimating clock rates from a two-level DLL according to an embodiment of the present disclosure, and Figure 10An example of an apparatus for audio channel synchronization according to an embodiment of the present disclosure is illustrated schematically. Detailed Implementation

[0030] The accompanying drawings and the following description are related to preferred embodiments only by way of illustration. It should be noted that, based on the discussion below, alternative embodiments of the structures and methods disclosed herein will be readily recognized as feasible alternatives that can be adopted without departing from the claimed principles.

[0031] Reference will now be made in detail to several embodiments, examples of which are illustrated in the accompanying drawings. It should be noted that similar or identical reference numerals may be used in the drawings where feasible, and these reference numerals may indicate similar or identical functions. The drawings depict embodiments of the disclosed system (or method) for illustrative purposes only. Those skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles described herein.

[0032] audio playback chain latency Figure 1 The diagram illustrates a system for processing and synchronizing audio playback across multiple playback chains, including local and remote audio playback paths.

[0033] The system includes a software stack (SW Stack) that provides audio signals with multiple audio channel signals. An audio channel signal is an audio signal designed to be played back along with other audio channel signals in a multi-channel audio signal. Different audio channel signals can then be transmitted through different processing and playback paths.

[0034] exist Figure 1 In the illustrated example, the audio system includes local and remote audio playback chains. In the local playback chain, the audio channel signal is routed directly to the DAC, then through the amplifier AMP, which ultimately drives the speaker. The delay introduced in this process is referred to as the local presentation delay in the following description.

[0035] In a remote playback chain, the audio channel signal is transmitted wirelessly via Bluetooth (Bluetooth Low Energy LE Audio), introducing additional transmission latency. After receiving the signal, it may undergo additional processing (e.g., decoding) before reaching the speaker via a DAC and AMP. The total delay from receiving the audio channel signal from the remote device to playback by the speaker is denoted as the remote presentation delay.

[0036] Figure 1The text highlights potential synchronization issues caused by transmission latency and varying presentation latency (remote and local), where remote audio playback may lag behind local playback. This issue can be particularly relevant in multi-speaker environments such as wireless home theater systems, distributed audio setups, and Bluetooth-enabled sound synchronization solutions.

[0037] Although Figure 1 Two different playback chains for two audio channel signals can be described, but this disclosure should not be construed as limited to two audio channel signals. The synchronization methods and systems described below can be used in combination with any number of audio channel signals (e.g., 7 channels), and any number of channel signals can be assigned to local and remote playback chains (e.g., 2 audio channel signals can be assigned to the local playback chain and 5 audio channel signals to the remote playback chain). Furthermore, although Bluetooth LE audio is used as an example of a remote playback chain, the synchronization methods and systems in this application can be used with any wireless technology suitable for transmitting audio signals (e.g., Wi-Fi).

[0038] For some wireless technologies, the signal providing the total latency of the replay chain may be unavailable or inaccurate. In these cases, it may be necessary to add additional timers for specific events in the replay chain to determine the total latency of the corresponding replay chain.

[0039] Combination Figure 2 and Figure 3 An example determination of the total latency of the Bluetooth LE audio playback chain is provided.

[0040] Figure 2 The illustration shows the time series of audio data units (SDUs) used for processing and transmitting in the Bluetooth LE audio playback chain. The timeline depicts the key delays introduced during the preparation and transmission of the audio channel signal, highlighting several timestamps that may be necessary to determine the corresponding delays.

[0041] This sequence begins when the host device writes the SDU, thus initiating the data transmission process. The bus time *to* represents the delay associated with transmitting the SDU from the host to the controller. Following this, preparation time is used for internal processing (e.g., encoding) before transmission over the Bluetooth wireless channel.

[0042] Once the Bluetooth controller transmits the SDU, it references a synchronization event, an SDU reference, and introduces a Connection Isochronous Group (CIG) synchronization delay to align data transmission with the Bluetooth isochronous interval. The CIG synchronization event establishes a common time reference for synchronized audio streams between multiple receivers.

[0043] Subsequently, the data packet undergoes a flush timeout period, defined as (FT-1) × ISO_Interval, which determines the maximum allowed latency before the packet is discarded. Finally, the audio signal is processed by the remote Bluetooth device for playback by the speaker. The latency introduced by this type of processing can be... Figure 1 The distant end depicted in the image is delayed.

[0044] According to the current Bluetooth LE audio specification (i.e., Bluetooth 6.0), two delays (i.e., bus time t0 and preparation time t1) may not be available or can not be determined.

[0045] For preparation time t1, the current Bluetooth specification has commands. HCI_LE_Read_ISO_TX_Sync The system takes a snapshot of the current SDU reference time. However, the SDU's reception time and the Bluetooth controller may not be available, making it impossible to determine the preparation time t1. To overcome this issue, a new timestamp is introduced in the Bluetooth controller.

[0046] Specifically, when the SDU arrives at the Bluetooth controller, a new timestamp can be obtained. T recv Then, for each SDU, the preparation time can be calculated as follows: T p = T recv T sdu ,in, T sdu This indicates the current SDU reference time. Then, it can be provided for each SDU. T p To expand HCI_Number_Of_Completed_Packets Events. Tables 1 and 2 show example extensions.

[0047] Table 1

[0048] For each Connection_Handle[i], the size of Preparation_Time[i][k] is Num_Completed_Packets[i] × 2 octets.

[0049] Table 2

[0050] Furthermore, Figure 3 The diagram illustrates the bus time t0 in the Bluetooth LE audio playback chain, representing the delay from when the host sends the SDU to when the Bluetooth controller receives the SDU.

[0051] In modern operating systems, applications should be readily available. T _write Other timestamps may be harder to obtain, for example... T _trigger (When data transmission begins) T _cmplt (When data transmission is complete) and T_ recv (When the Bluetooth controller receives data). This availability may depend on the specific driver / kernel implementation, the system's runtime load, and the host kernel. T_ write and T_ trigger The buffering situation between them.

[0052] To achieve higher latency calculation accuracy, the host should be as close as possible to... T_ cmplt The Bluetooth controller should timestamp the SDU data packet immediately after data transmission is complete and provide this information to the application. T_ cmplt and T_ recv The elapsed time is negligible. Therefore, the bus time t0 can be calculated as... T_ cmplt - T _write .

[0053] Although they have already combined Figure 2 and Figure 3 Tables 1 and 2 illustrate examples for determining missing delays in a Bluetooth LE audio playback chain; however, these examples should not be construed as limiting the synchronization methods and systems according to embodiments of this application. Different timestamps or corresponding delays can be determined in different ways. Synchronization of signals from different audio channels is possible as long as the complete delay can be transmitted to the synchronization system.

[0054] Regarding local rendering latency, some latency can be fixed during hardware design, while other latency can be captured via timestamps. This may depend on the specific implementation of the local playback chain.

[0055] Synchronization system Next, a synchronization system will be described that uses a determined delay to synchronize different audio channel signals so that these audio channel signals are synchronously played back by the speakers of the corresponding audio playback chain.

[0056] Figure 4A schematic diagram illustrates a synchronization system comprising two synchronization subsystems. Regardless of the specific audio playback chain in which the synchronization subsystems are used, each synchronization subsystem will likely operate in the same way. Therefore, the functionality of the synchronization subsystems will only be relevant to... Figure 4 The upper synchronization subsystem 100 is described herein. Again, note that the number of synchronization subsystems / audio playback chains and the types of audio playback chains are merely examples to illustrate the functionality of the synchronization system. Any number and type of audio playback chains can be combined with the synchronization system, provided the number of synchronization subsystems equals the number of different audio playback chains. The synchronization system can be implemented entirely in hardware, software, or a combination of both.

[0057] Synchronization subsystem 100 may include delay-locked loop DLL 101. DLL 101 may receive a delay signal from an audio playback chain (e.g., from a Bluetooth LE audio playback chain). The delay signal may correspond to the total delay experienced by the audio channel signal during transmission through the audio playback chain, for example, Figure 2 The total Bluetooth LE audio delay is described in the diagram. The delay signal can be a time-varying signal, meaning it can report the total delay at fixed intervals (e.g., on the order of milliseconds). The goal of DLL 101 can be to keep the delay of the corresponding audio playback chain substantially constant over time. In particular, the total delay of the audio playback chain can depend on the clock signal used to drive the digital components (e.g., DAC) of the audio playback chain. Therefore, the delay of the audio playback chain can be adjusted by the clock rate of the clock signal. To control the delay of the audio playback chain, DLL 101 can determine a difference signal between the current value of the delay signal and a reference delay signal. The reference delay can be a reference delay for the corresponding audio playback chain. For example, the reference delay can be the initial total delay reported by the audio playback chain after startup. The difference between the current value of the delay signal and the reference delay can be represented as an error signal. The error signal is fed into the controller of DLL 101. The controller of DLL 101 can be a function that determines a target clock rate for the digital portion of the audio playback chain such that the error signal is reduced when the target clock rate is applied to the audio playback chain. The controller can be any suitable controller that implements this function, such as a proportional-integral-derivative (PID) controller. PID controllers can be tuned using known tuning methods.

[0058] Therefore, the output of DLL 101 can be a clock rate. Furthermore, DLL 101 can also output a delayed signal received from the corresponding audio playback link.

[0059] The clock rate can be the input to the phase-locked loop (PLL) 102. The PLL 102 smooths the clock rate generated by the DLL 101. Specifically, the PLL 102 serves as a feedback control system that continuously tracks, corrects, and stabilizes its input clock signal by dynamically adjusting the output frequency of the DLL 101. Using the PLL 102 reduces short-term timing fluctuations in the clock signal. Any suitable PLL can be used as the PLL 102. Since the output signal of the PLL 102 will be used to resample the audio channel signal, it will be referred to as the sampling rate in the following description.

[0060] Furthermore, the synchronization subsystem 100 may include a resampler 103. The resampler 103 can receive audio channel signals to be transmitted via the corresponding audio playback chain. The audio channel signals can be sampled according to a predefined frequency. The predefined frequency may, for example, depend on the codec used in the underlying technology of the corresponding audio playback chain. For example, Bluetooth LE audio can use the LC3 codec. Therefore, audio channel signals intended for use in a Bluetooth LE playback audio chain can be sampled according to the LC3 codec.

[0061] An audio channel signal can include multiple audio sub-channel signals. In other words, multiple audio sub-channel signals can use the same type of audio playback chain.

[0062] The resampler 103 can receive the sampling rate output by the PLL 102 and resample the audio channel signal from the original sampling rate to the received sampling rate, so that the sampling rate of the audio channel signal after the resampler 103 matches the sampling rate of the DAC in the corresponding audio playback chain. This enables smooth operation of the DAC when converting the audio channel signal to the analog domain.

[0063] Therefore, the resampler 103 can output a resampled audio channel signal (resampled according to the sampling rate determined by the PLL 102) and provide the resampled audio channel signal to the delay line element 104.

[0064] Delay line element 104 can also receive a delay signal from DLL 101. The goal of delay line element 104 is to introduce a delay into the resampled audio channel signals so that different audio channel signals are synchronously played back by the speakers of the corresponding audio playback chain. This can be achieved in different ways. For example, delay line element 104 of each synchronization subsystem can have the same target delay and delay the signal such that the sum of the additional delay and the current value of the delayed signal equals the target delay. Thus, each audio channel signal can be delayed such that the experienced delay corresponds to the target delay.

[0065] Alternatively, the delayed signal can be transmitted between different synchronization subsystems. Thus, it may only be necessary to delay the audio channel signal corresponding to the fast audio playback chain (e.g., a local audio playback chain). Therefore, the total delay for each audio playback chain may be determined by the slowest audio playback chain.

[0066] Finally, DLL 101 can use the determined clock rate to determine the rate at which data from the delayed and resampled audio channel signals is fed to the interface of the corresponding audio playback chain. The type of interface can depend on the type of audio playback chain. For example, for a local audio playback chain, the interface could be the DAC of the audio playback chain. For a remote audio playback chain, the interface could be located on the host of the system, such as a desktop CPU, TV, mobile phone, or any other device that can initiate a Bluetooth connection.

[0067] In summary, by implementing a synchronization subsystem 100 for each audio playback chain, synchronized playback can be achieved even if the audio channel signal passes through completely different types of audio playback chains.

[0068] Improved DLL Depending on the "quality" of the delayed signal, a DLL (such as DLL 104) may fail to converge to a stable clock rate. In particular, the delayed signal may be too coarse. For example, the delayed signal might be quantized to a level that is too coarse for the DLL. This can happen when timestamps are generated by very low-resolution timers in an audio playback chain. When such a delayed signal is input to a DLL, the DLL may never converge to a stable clock rate and instead oscillate around the clock rate it is trying to match.

[0069] To enable convergence to the clock rate in these conditions, a two-level DLL is proposed.

[0070] Figure 5 An example of a two-level DLL 200 is illustrated. A two-level DLL 200 may include a first DLL level 210 and a second DLL level 220. When DLL 200 is initialized, only the first DLL level 210 can run. Specifically, the first DLL level 210 can receive information such as... Figure 4 The described delay signal. Based on the delay signal and the reference delay, a first error signal is determined and fed to the controller of the first DLL stage 210 to estimate the first clock rate. As previously mentioned, the first DLL stage 210 may not converge to a stable clock rate.

[0071] In addition to outputting the first clock rate, DLL stage 210 can also transmit the first error signal and the first clock rate to histogram accumulator 230. Histogram accumulator 230 can accumulate pairs of the first error signal and the first clock rate, i.e., pairs of values ​​of the first error signal and the first clock rate at specific points in time. When a certain threshold is reached, the histogram can be determined using the pairs of the first error signal and the first clock rate. This threshold can correspond to the number of pairs of the first error signal and the first clock rate. Alternatively, this threshold can be time-based, i.e., histogram accumulator 230 can accumulate pairs of the first error signal and the first clock rate over a predefined time period. When the threshold is reached, histogram accumulator 230 can determine the histogram based on the accumulated pairs of the first error signal and the first clock rate. Figure 7 The upper part of the diagram illustrates an example of this histogram. In this example, It can represent the first clock rate, and This can represent the first error signal, where, This represents the bin index in the histogram. The number and size of bins can be predefined or depend on the actual values ​​of the first error signal and the first clock rate. Furthermore, the number and size of bins can depend on the type of audio playback chain.

[0072] Histogram accumulator 230 can then output the histogram to confidence-based estimator 240. Confidence-based estimator 240 can use the histogram to estimate the second clock rate, where the second clock rate may correspond to the case where the error signal is equal to zero. Preferably, confidence-based estimator 240 can use only the most frequent pairs of values ​​in the histogram to exclude outliers in the confidence-based estimate of the second clock rate. Specifically, confidence-based estimator 240 can use the N most frequent pairs of the first error signal and the first clock rate, where N is a positive integer. The estimation of the second clock rate can be performed, for example, by using linear regression. Figure 7 The lower part of the diagram illustrates an example of estimating the second clock rate based on linear regression. Based on the histogram accumulator 230 and the confidence-based estimator 240, the second clock rate corresponding to a zero error signal can be estimated even if the first clock rate output by the first DLL stage 210 oscillates around the second clock rate.

[0073] After estimating the second clock rate, the first DLL stage 210 can stop running, and the second clock rate is transmitted to the second DLL stage 220.

[0074] The delayed signal can now be used as input to the second DLL stage 220 to estimate the second clock rate. The DLL stage 220 can determine a second error signal corresponding to the difference between the delayed signal and a delayed version of the delayed signal. This second error signal can then be used as input to the controller of the second DLL stage 220, which can also receive the second clock rate. By using the second error signal and the second clock rate as inputs, the controller of the second DLL stage 220 may be able to converge to a stable third clock rate. The DLL stage 220 can then output the third clock rate along with the delayed signal to the synchronization subsystem 100.

[0075] Furthermore, Figure 6 An example implementation of a two-level DLL 200 is illustrated. In this example implementation, the first DLL level 210 may include a window averaging block 211 and a proportional-derivative (PD) controller 212. The window averaging block 211 may perform an averaging function on the delayed signal. Specifically, the delayed signal may be time-averaged by the window averaging block 211 to generate a first averaged delay signal, and a first error signal may be determined based on the first averaged delay signal and a reference delay. The PD controller 212 then determines / estimates a first clock rate based on the first error signal. The determination / estimation of the first clock rate may further be based on a reference clock signal. .

[0076] The second DLL level 220 may include a window averaging block 221 and a proportional P controller 222. Similar to window averaging block 211, window averaging block 221 may also perform an averaging function. The output of window averaging block 221 may be a second averaged delay signal. However, the average window of window averaging block 221 may be longer than that of window averaging block 211. For example, the average window of window averaging block 221 may be 1 minute, while the average window of window averaging block 211 may be 1 second. The second error function may be determined based on the second averaged delay function and a delayed version of the second averaged delay function.

[0077] The P controller 222 can estimate / determine the third clock rate based on the second error function and the second clock rate already determined by the confidence-based estimator 240.

[0078] By using a two-level DLL 200, a stable clock rate can be estimated even if the input delay signal fluctuates too much to be estimated by a single DLL level.

[0079] Based on the above, a method 300 for synchronizing two or more audio channel signals is provided, such as... Figure 8 The flowchart depicts this. In addition to the following method steps, method 300 may optionally include the steps outlined above regarding the already combined... Figures 4 to 7 The described synchronization subsystem 100 and the two-level DLL 200 are all variations described. Method 300 can be performed for each audio channel signal, wherein each audio channel signal is intended for a separate audio playback chain.

[0080] exist Step S301 In this process, the system receives the delay signal from the corresponding audio playback chain. The audio playback chain can be a local or remote (wireless) audio playback chain. The delay signal can represent the total delay experienced by the audio channel signal as it is processed by the audio playback chain (e.g., from the interface of the audio playback chain to the speaker output of the audio playback chain).

[0081] exist Step S302 In this context, the DLL estimates / determines the clock rate based on the delayed signal. The DLL can estimate / determine the clock rate so that when the clock rate is applied to the audio channel signal, the delayed signal can remain essentially constant.

[0082] exist Step S303 In this context, the PLL can estimate / determine the sampling rate based on the clock rate.

[0083] exist Step S304 In this process, the corresponding audio channel signal is resampled based on the sampling rate to generate a resampled audio channel signal. The original sampling rate of the audio channel signal can be related to the codec used for the audio channel signal, while the estimated / determined sampling rate can correspond to the sampling rate of the DAC in the corresponding audio playback chain.

[0084] exist Step S205 In this process, the audio channel signals are resampled based on the delayed signal to generate delayed resampled audio channel signals, ensuring that two or more delayed resampled audio channel signals remain time-synchronized during playback. The synchronization delay can be based on a target delay.

[0085] Figure 9 A method 400 for determining / estimating the clock rate based on a delayed signal via a two-stage DLL is shown. For example, method 400 may implement step S302.

[0086] exist Step 401 In this process, a delayed signal is received. The delayed signal corresponds to a signal delay on the signal transmission chain. The signal transmission chain may include digital components. For example, the signal transmission chain may be the audio playback chain in method 300.

[0087] exist Step 402In this process, the first DLL level estimates / determines a first clock rate based on a first error signal between the delayed signal and a reference delay. The first clock rate is estimated such that when the first clock rate is applied to the signal fed into the signal transmission chain, the delayed signal remains substantially constant over time. The first clock rate may be the output of method 400 after initializing the two-level DLLs.

[0088] exist Step 403 In this process, the first clock rate and the first error signal are accumulated over time until a threshold is reached. The threshold can correspond to the number of pairs of the first clock rate and the first error signal that have been accumulated.

[0089] exist Step 404 In the process, after reaching a threshold, a second clock rate is estimated based on the accumulated pairs. The second clock rate corresponds to a first error signal that is equal to zero.

[0090] exist Step 405 In this process, delayed signal processing is switched from the first DLL level to the second DLL level. In other words, the first DLL stops running and no longer outputs the first clock rate, while the second DLL can perform clock rate estimation.

[0091] exist Step 406 In the second LL, a third clock rate is estimated / determined based on the delayed signal and the second clock rate. The third clock rate is estimated such that when the third clock rate is applied to the signal fed into the signal transmission chain, the delayed signal remains substantially constant over time. Therefore, the third clock rate can be determined by method 400 in... Step 405 The output after the switching operation.

[0092] Although methods for synchronizing audio channel signals and estimating clock rates have been described above, this disclosure also relates to corresponding apparatus. Reference will be made below. Figure 10 Description of embodiments providing such a device.

[0093] like Figure 10 As shown, the device 500 includes a processor 50 and a memory 502. The memory 502 is configured to store program code. The processor 501 is configured to execute instructions in the program code, causing the device 500 to perform audio channel signal synchronization as described in any of the above embodiments and implementations. Depending on the use case and / or implementation, the processor 501 may also receive suitable input data (e.g., audio channel signals and delay signals), among other things. Depending on the use case and / or implementation, the processor 501 may be adapted to perform the methods / techniques described throughout this disclosure (e.g., referred to above respectively). Figure 8 and Figure 9Methods 300 and 400 are described, and corresponding output data (e.g., resampled audio channel signals with clock rate and delay) is generated. This device can be part of a device used for orchestrating audio playback (e.g., a personal computer, TV, or smart speaker).

[0094] The aspects of the system described herein can be implemented in a suitable computer-based audio processing network environment to process digital or digitized audio files. Parts of the adaptive audio system may include one or more networks comprising any desired number of independent machines, including one or more routers (not shown) for buffering and routing data transmitted between computers. Such networks can be built on a variety of different network protocols and can be the Internet, a wide area network (WAN), a local area network (LAN), or any combination thereof.

[0095] One or more components, blocks, processes, or other functional units may be implemented by a computer program executed by a processor-based computing device controlling the system. It should also be noted that any number of combinations of hardware, firmware, and / or data and / or instructions embodied in various machine-readable or computer-readable media may be used to describe the various functions disclosed herein in terms of their behavior, register transfers, logical components, and / or other characteristics. Computer-readable media that may embody such formatted data and / or instructions include, but are not limited to, various forms of physical (non-transitory), non-volatile storage media, such as optical, magnetic, or semiconductor storage media.

[0096] While one or more embodiments have been described by way of example and specific examples, it should be understood that one or more embodiments are not limited to the disclosed embodiments. Rather, they are intended to cover various modifications and similar arrangements that will be obvious to those skilled in the art. Therefore, the scope of the appended claims should be given the broadest interpretation in order to cover all such modifications and similar arrangements.

[0097] explain Computing devices implementing the techniques described above can have the following example architectures. Other architectures are also possible, including those with more or fewer components. In some implementations, the example architecture includes one or more processors (e.g., a dual-core Intel® Xeon® processor), one or more output devices (e.g., an LCD), one or more network interfaces, one or more input devices (e.g., a mouse, keyboard, touch-sensitive display), and one or more computer-readable media (e.g., RAM, ROM, SDRAM, hard disk, optical disk, flash memory, etc.). These components can exchange communication and data via one or more communication channels (e.g., buses), which can utilize various hardware and software to facilitate the transfer of data and control signals between the components.

[0098] The term "computer-readable medium" refers to a medium that participates in providing instructions to a processor for execution, including but not limited to non-volatile media (e.g., optical discs or magnetic disks), volatile media (e.g., memory), and transmission media. Transmission media include, but are not limited to, coaxial cables, copper wires, and optical fibers.

[0099] Computer-readable media may further include an operating system (e.g., a Linux® operating system), a network communication module, an audio interface manager, an audio processing manager, and a real-time content distributor. The operating system may be multi-user, multi-processing, multi-tasking, multi-threaded, real-time, etc. The operating system performs basic tasks, including but not limited to: identifying input from network interfaces and / or devices and providing output to network interfaces and / or devices; recording and managing files and directories on computer-readable media (e.g., memory or storage devices); controlling peripheral devices; and managing traffic on one or more communication channels. The network communication module includes various components (e.g., software for implementing communication protocols such as TCP / IP and HTTP) for establishing and maintaining network connections.

[0100] The architecture can be implemented in parallel processing or peer-to-peer infrastructure, or on a single device with one or more processors. The software may include multiple software components or may be a single code body.

[0101] The described features can be advantageously implemented in one or more computer programs that can execute on a programmable system including at least one programmable processor coupled to receive and transfer data and instructions from a data storage system, at least one input device, and at least one output device. A computer program is a set of instructions that can be used directly or indirectly in a computer to perform an activity or produce a result. Computer programs can be written in any form of programming language (e.g., Objective-C, Java), including compiled or interpreted languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, browser-based web application, or other unit suitable for use in a computing environment.

[0102] For example, suitable processors for executing instruction programs include both general-purpose microprocessors and special-purpose microprocessors, as well as a single processor or one or more processors or cores in any type of computer. Typically, the processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor for executing instructions and one or more memories for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data files, or operatively coupled to and communicating with them; such devices include disks, such as internal hard disks and removable disks; magneto-optical disks; and optical disks. Storage devices suitable for tangibly representing computer program instructions and data include all forms of non-volatile memory, such as semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory can be supplemented by or incorporated into an ASIC (Application-Specific Integrated Circuit).

[0103] To provide user interaction, features can be implemented on a computer having a display device for showing information to the user, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor or retina display device. The computer may have touch surface input devices (e.g., touchscreens) or a keyboard and pointing devices such as a mouse or trackball, through which the user can provide input to the computer. The computer may also have a voice input device for receiving voice commands from the user.

[0104] The features can be implemented in a computer system that includes back-end components, such as data servers, or middleware components, such as application servers or internet servers, or front-end components, such as client computers with graphical user interfaces or internet browsers, or any combination thereof. The components of the system can be connected by any form or medium of digital data communication, such as a communication network. Examples of communication networks include, for example, LANs, WANs, and computers and networks that form the internet.

[0105] A computing system may include clients and servers. Clients and servers are typically geographically separated and usually interact via a communication network. The client-server relationship arises from computer programs running on respective computers and having a client-server relationship relative to each other. In some embodiments, the server transmits data (e.g., HTML pages) to the client device (e.g., to display data to a user interacting with the client device and to receive user input from the user). Data generated at the client device (e.g., the result of user interaction) can be received from the client device at the server.

[0106] A system of one or more computers can be configured to perform specific actions by means of software, firmware, hardware, or a combination thereof installed on the system that will cause the system to perform actions in operation. One or more computer programs can be configured to perform specific actions by means of instructions that, when executed by a data processing device, cause the device to perform those actions.

[0107] While this specification contains numerous specific implementation details, these details should not be construed as limiting the scope of any invention or claim, but rather as descriptions of features characteristic of particular embodiments of a particular invention. Specific features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as functioning in a particular combination and even initially claimed, in some cases one or more features from the claimed combination may be removed from the combination, and the claimed combination may be for sub-combinations or variations thereof.

[0108] Similarly, although operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring the operations to be performed in the specific order shown or sequentially, or to perform all illustrated operations to achieve the desired result. In some cases, multitasking and parallel processing can be advantageous. Furthermore, the separation of various system components in the embodiments described above should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0109] Unless otherwise specifically stated, it is obvious from the following discussion that, throughout the discussion of the invention, terms such as “processing,” “computing,” “calculating,” “determining,” and “analyzing” are used to refer to the actions and / or processes by which a computer or computing system or similar electronic computing device manipulates and / or transforms data represented as physical (e.g., electronic) quantities into other data similarly represented as physical quantities.

[0110] Throughout the invention, references to "one exemplary embodiment," "some exemplary embodiments," or "exemplary embodiment" mean that a particular feature, structure, or characteristic described in connection with an exemplary embodiment is included in at least one exemplary embodiment of the invention. Therefore, the phrases "in one exemplary embodiment," "in some exemplary embodiments," or "in an exemplary embodiment" appearing throughout the invention do not necessarily refer to the same exemplary embodiment. Furthermore, in one or more exemplary embodiments, specific features, structures, or characteristics can be combined in any suitable manner, as will be apparent to those skilled in the art according to the invention.

[0111] As used herein, unless otherwise specified, ordinal adjectives such as “first,” “second,” “third,” etc., are used to describe common objects only to indicate different instances of similar objects and are not intended to imply that the objects described must be in a given order in time, space, hierarchy, or any other way.

[0112] Similarly, it should be understood that the wording and terminology used herein are for descriptive purposes and should not be considered restrictive. The use of “including,” “comprising,” or “having,” and variations thereof, is intended to cover the items listed thereafter and their equivalents, as well as additional items. Unless otherwise specified or limited, the terms “installation,” “connection,” “support,” and “coupled,” and variations thereof, are used extensively and cover direct and indirect installation, connection, support, and coupling.

[0113] In the claims below and in the description herein, the terms *comprising*, *comprised of*, or *which comprises* are open-ended terms meaning that at least the following element / feature is included, but not excluding other elements / features. Therefore, when the term *comprising* is used in a claim, it should not be construed as limited to the devices, elements, or steps listed thereafter. For example, the scope of the expression *devices including A and B* should not be limited to devices that only include elements A and B. As used herein, the terms *including*, *which includes*, or *that includes* are also open-ended terms meaning that at least the element / feature following the term is included, but not excluding other elements / features. Therefore, *including* is synonymous with *comprising* and means *comprising*.

[0114] It should be understood that in the above description of exemplary embodiments of the invention, various features of the invention are sometimes combined together in a single exemplary embodiment / figure or its description in order to simplify the invention and aid in understanding one or more of the inventive aspects. However, the method of the invention should not be construed as reflecting an intention in the claims to require more features than those expressly recited in each claim. Rather, as reflected in the following claims, the inventive aspects lie in fewer than all features of a single foregoingly disclosed exemplary embodiment. Therefore, the claims following this specification are hereby expressly incorporated, wherein each claim is an independent exemplary embodiment of the invention.

[0115] Furthermore, while some exemplary embodiments described herein include features that are included in other exemplary embodiments but not others, as those skilled in the art will understand, combinations of features from different exemplary embodiments are intended to be within the scope of the invention and to form different exemplary embodiments. For example, any exemplary embodiment of the claimed embodiments in the appended claims can be used in any combination.

[0116] Numerous specific details are set forth in the description provided herein. However, it should be understood that exemplary embodiments of the invention may be practiced without these specific details. In other instances, well-known methods, structures, and techniques have not been shown in detail in order to avoid obscuring the understanding of this specification.

[0117] Therefore, although the mode considered to be the best mode of the invention has been described, those skilled in the art will recognize that other and further modifications can be made thereto without departing from the spirit of the invention, and all such changes and modifications falling within the scope of the invention are intended to be claimed. For example, any formulas given above merely indicate procedures that can be used. Functions can be added or removed from the block diagrams, and operations can be interchanged within functional blocks. Steps can be added or removed from the methods described within the scope of this disclosure.

[0118] Various aspects and implementations of this disclosure can also be understood from the following enumerated exemplary embodiments (EEE), which are not claims.

[0119] EEE 1. A method for synchronizing two or more audio channel signals, each audio channel signal being processed by a separate audio playback chain, the method comprising, for each audio channel signal: Receive the delay signal of the corresponding audio playback chain; Based on the delay signal, the clock rate is estimated using a delay-locked loop (DLL). Based on the clock rate, the sampling rate is estimated using a phase-locked loop (PLL). The corresponding audio channel signal is resampled based on the sampling rate to generate a resampled audio channel signal; and The resampled audio channel signal is delayed based on the delayed signal to generate a delayed resampled audio channel signal, so that the two or more delayed resampled audio channel signals remain time-synchronized during playback.

[0120] EEE 2. The method according to EEE 1, wherein the clock rate is estimated such that when the clock rate is applied to the audio channel signal, the delay signal remains substantially constant over time.

[0121] EEE 3. The method according to EEE 1 or 2, wherein the method further comprises: The clock rate is applied to the delayed resampled audio channel signal to set the rate at which samples of the delayed resampled audio channel signal are transmitted through the corresponding audio playback chain.

[0122] EEE 4. The method according to EEE 3, wherein setting the rate at which samples of the delayed resampled audio channel signal are transmitted through the corresponding audio playback chain comprises: providing the samples to the audio interface of the corresponding audio playback chain at the set rate.

[0123] EEE 5. The method according to any of the foregoing EEEs, wherein estimating the clock rate using the DLL based on the delayed signal comprises: Based on a first error signal between the delayed signal and a reference delay, a first clock rate is estimated using a first DLL level of the DLL, wherein the first clock rate is estimated such that when the first clock rate is applied to the audio channel signal, the delayed signal remains substantially constant over time; The pair of the first clock rate and the first error signal is accumulated over time until a threshold is reached; After the threshold is reached, the second clock rate corresponding to the first error signal equal to zero is estimated based on the accumulated pairs. Switching the delayed signal processing from the first DLL level of the DLL to the second DLL level of the DLL; and Based on the delayed signal and the second clock rate, a third clock rate is estimated using the second DLL level, wherein the third clock rate is estimated such that when the third clock rate is applied to the signal fed to the audio channel signal, the delayed signal remains substantially constant over time; Specifically, before switching the delayed signal processing from the first DLL level to the second DLL level, the clock rate corresponds to the first clock rate; and after switching the delayed signal processing from the first DLL level to the second DLL level, the clock rate corresponds to the third clock rate.

[0124] EEE 6. The method according to any of the preceding EEEs, wherein the clock rate corresponds to the conversion rate of the digital-to-analog converter (DAC) of the corresponding audio playback chain.

[0125] EEE 7. The method according to any of the preceding EEEs, wherein each audio channel signal comprises multiple audio sub-channel signals processed by the same audio playback chain.

[0126] EEE 8. The method according to any of the preceding EEEs, wherein each audio playback chain is one of a local audio playback chain or a remote audio playback chain.

[0127] EEE 9. The method according to EEE 8, wherein the remote audio playback chain includes one of a Bluetooth Low Energy audio playback chain or a WiFi audio playback chain.

[0128] EEE 10. The method according to EEE 8 or 9, wherein the delay signal of the corresponding audio playback chain indicates an audio playback delay, and if the corresponding audio playback chain is a local audio playback chain, the audio playback delay includes a local presentation delay of the audio signal; or if the corresponding audio playback chain is a remote audio playback chain, the audio playback delay includes a transmission delay of the audio signal over the wireless medium and a remote presentation delay of the audio signal.

[0129] EEE 11. The method according to EEE 10, wherein the local presentation delay is the delay from the audio signal entering the digital-to-analog converter (DAC) to being played back by the speaker, and wherein the far-end presentation delay is the delay from the audio signal being received by the far end portion of the far-end audio playback chain to be played back by the speaker.

[0130] EEE 12. The method according to any of the preceding EEEs, wherein estimating the sampling rate using the PLL based on the clock rate comprises: smoothing the clock rate to estimate the sampling rate.

[0131] EEE 13. The method according to any of the preceding EEEs, wherein resampling the corresponding audio channel signal based on the sampling rate converts the source digital audio sampling rate of the audio channel signal to the sampling rate.

[0132] EEE 14. The method according to any of the preceding claims, wherein generating the delayed resampled audio channel signals such that the two or more delayed resampled audio channel signals remain time-synchronized during playback comprises: delaying the resampled audio channel signals according to the difference between a preset delay and the delayed signals, wherein the preset delay is equal for all audio channel signals.

[0133] EEE 15. A method for estimating clock rate based on a delayed signal using a two-stage delay-locked loop (DLL), the method comprising: Receive the delayed signal, wherein the delayed signal corresponds to the signal delay on the signal transmission chain; Based on a first error signal between the delayed signal and a reference delay, a first clock rate is estimated using a first DLL level, wherein the first clock rate is estimated such that when the first clock rate is applied to a signal fed into the signal transmission chain, the delayed signal remains substantially constant over time. The pair of the first clock rate and the first error signal is accumulated over time until a threshold is reached; After the threshold is reached, the second clock rate corresponding to the first error signal equal to zero is estimated based on the accumulated pairs. Switch the delayed signal processing from the first DLL level to the second DLL level; and Based on the delayed signal and the second clock rate, a third clock rate is estimated using the second DLL level, wherein the third clock rate is estimated such that when the third clock rate is applied to the signal fed to the signal transmission chain, the delayed signal remains substantially constant over time.

[0134] EEE 16. The method according to EEE 15, wherein estimating the first clock rate using the first DLL level includes: A first averaging function is applied to the delayed signal to generate a first average delayed signal; Determine the first error signal between the first average delay signal and the reference delay; and The first clock rate is estimated by the proportional-derivative controller based on the error signal.

[0135] EEE 17. The method according to EEE 16, wherein the reference delay is a delay acquired during the start-up of the signal transmission chain.

[0136] EEE 18. The method according to EEE 16 or 17, wherein the clock rate is estimated by the proportional derivative controller based on a reference clock rate.

[0137] EEE 19. The method according to any one of EEE 15 to 18, wherein the threshold corresponds to the number of pairs of the accumulated first clock rate and the difference signal.

[0138] EEE 20. The method according to any one of EEE 15 to 19, wherein the estimation of the second clock rate corresponding to the first error signal equal to zero based on the accumulated values ​​comprises: Generate a histogram of the pair of the first clock rate and the first error signal; and The second clock rate is estimated based on the pair of the N highest frequencies of the first clock rate in the histogram and the first error signal, where N is an integer greater than zero.

[0139] EEE 21. The method according to EEE 20, wherein estimating the second clock rate based on the pair of the N highest-frequency first clock rates in the histogram and the first error signal is performed by linear regression.

[0140] EEE 22. The method according to any one of EEE 15 to 21, wherein estimating the third clock rate using the second DLL level comprises: A second averaging function is applied to the delayed signal to generate a second averaged delayed signal; Determine a second error signal between the second average delay signal and a delayed version of the second average delay signal; and The third clock rate is estimated by the proportional controller based on the second error signal and the second clock rate.

[0141] EEE 23. According to the method of EEE 22, when EEE 22 belongs to any one of EEE 16 to 18, wherein the average window of the first average function is shorter than the average window of the second average function.

[0142] EEE 24. The method according to EEE 23, wherein the length of the average window of the first averaging function is equal to 1 second, and the length of the average window of the second averaging function is equal to 1 minute.

[0143] EEE 25. An apparatus comprising one or more processors and a memory, the memory being coupled to the one or more processors and storing instructions for the one or more processors, wherein the one or more processors are adapted to perform a method according to any one of EEE 1 to 24.

[0144] EEE 26. A computer program comprising instructions that, when executed by one or more processors, cause the one or more processors to perform the method according to any one of EEE 1 to 24.

[0145] EEE 27. A computer-readable storage medium storing a computer program according to EEE 26.

[0146] EEE 28. A system for synchronously playing back a first audio channel transmitted via a first audio codec and a second audio channel transmitted via a second audio codec, the system comprising: A first channel module, the first channel module comprising: A first channel resampler is configured to convert the source digital audio sampling rate of the first audio channel to a first digital-to-analog converter (DAC) sampling rate. A first channel phase-locked loop, configured to provide the first DAC sampling rate to the first channel resampler; The first channel delay-locked loop is configured as follows: Synchronize the first DAC sampling rate with the characteristics of the first audio codec; and At the first transmission reference point, a first delay is maintained for the first audio channel; The first channel delay line is configured as follows: The first audio channel is provided to the first audio interface at the first DAC sampling rate and the first delay; and The second channel module includes: A second channel resampler is configured to convert the source digital audio sample rate of the second audio channel to a second digital-to-analog converter (DAC) sample rate; A second channel phase-locked loop is configured to provide the second DAC sampling rate to the second channel resampler; The second channel delay-locked loop is configured as follows: Synchronize the second DAC sampling rate with the characteristics of the second audio codec; and At the second transmission reference point, a second delay is maintained for the second audio channel; The second channel delay line is configured as follows: The second audio channel is provided to the second audio interface at the second DAC sampling rate and the second delay.

[0147] EEE 29. The system as described in EEE 28 further includes: A plurality of first audio channels, wherein the plurality of first audio channels includes at least one first audio channel; and A plurality of second audio channels, wherein the plurality of second audio channels includes at least one second audio channel.

[0148] EEE 30. A system as described in EEE 28 or 29, wherein the first audio codec includes at least one of a Bluetooth LE audio codec or a local audio codec, and the second audio codec includes at least one of the Bluetooth LE audio codec or the local audio codec.

[0149] EEE 31. The system of any one of EEE 28 to 30, wherein the characteristic of the first audio codec includes a delay associated with the first audio codec, and wherein the characteristic of the second audio codec includes a delay associated with the second audio codec.

[0150] EEE 32. The system of any one of EEE 28 to 31, wherein the first delay or the second delay includes at least one of local rendering delay, remote rendering delay and / or transmission delay.

[0151] EE 33. The system of any one of EE 28 to 32, wherein the first audio interface or the second audio interface includes at least one of a Bluetooth LE audio interface and / or a local audio interface.

[0152] EEE 34. A delay-locked loop (DLL) system for determining conversion rate and delay, the delay-locked loop system comprising: The estimation level is configured as follows: Estimate the switching clock rate sample; Maintain a constant delay; Record the error between the constant delay and the switching clock rate sample in the histogram; Linear regression is performed based on the histogram to estimate one or more initial switching clock rate samples; The control level is used for: The conversion rate and the delay are determined based on one or more estimated initial conversion clock rate samples.

[0153] EEE 35. A delay-locked loop system as described in EEE 34, wherein the estimation stage includes a proportional-derivative controller.

[0154] EEE 36. A delay-locked loop system as described in EEE 34 or 35, wherein the control stage includes a proportional controller.

[0155] EEE 37. The delay-locked loop system as described in any one of EEE 34 to 37, wherein the linear regression is performed using the N data points with the highest frequency in the histogram.

[0156] EEE 38. A delay-locked loop system as described in EEE 37, wherein the data points include the error of the constant delay and / or the sample of the switching clock rate.

Claims

1. A method for synchronizing two or more audio channel signals, each audio channel signal being processed by a separate audio playback chain, the method comprising, for each audio channel signal: Receive the delay signal of the corresponding audio playback chain; Based on the delay signal, the clock rate is estimated using a delay-locked loop (DLL). Based on the clock rate, the sampling rate is estimated using a phase-locked loop (PLL). The corresponding audio channel signal is resampled based on the sampling rate to generate a resampled audio channel signal; as well as The audio channel signal is delayed and resampled based on the delayed signal to generate a delayed resampled audio channel signal, so that the two or more delayed resampled audio channel signals are synchronized in time during playback.

2. The method according to claim 1, wherein, The clock rate is estimated such that when the clock rate is applied to the audio channel signal, the delay signal remains substantially constant over time.

3. The method according to claim 1 or 2, wherein, The method further includes: The clock rate is applied to the delayed resampled audio channel signal to set the rate at which samples of the delayed resampled audio channel signal are transmitted through the corresponding audio playback chain.

4. The method according to claim 3, wherein, Setting the rate at which samples of the delayed resampled audio channel signal are transmitted through the corresponding audio playback chain includes providing the samples to the audio interface of the corresponding audio playback chain at the set rate.

5. The method according to any one of the preceding claims, wherein, Estimating the clock rate using the DLL based on the delayed signal includes: Based on a first error signal between the delayed signal and a reference delay, a first clock rate is estimated using a first DLL level of the DLL, wherein the first clock rate is estimated such that when the first clock rate is applied to the audio channel signal, the delayed signal remains substantially constant over time; The pair of the first clock rate and the first error signal is accumulated over time until a threshold is reached; After the threshold is reached, the second clock rate corresponding to the first error signal equal to zero is estimated based on the accumulated pairs. Switching the delayed signal processing from the first DLL level of the DLL to the second DLL level of the DLL; and Based on the delayed signal and the second clock rate, a third clock rate is estimated using the second DLL level, wherein the third clock rate is estimated such that when the third clock rate is applied to the signal fed to the audio channel signal, the delayed signal remains substantially constant over time; Specifically, before switching the delayed signal processing from the first DLL level to the second DLL level, the clock rate corresponds to the first clock rate; and after switching the delayed signal processing from the first DLL level to the second DLL level, the clock rate corresponds to the third clock rate.

6. The method according to any one of the preceding claims, wherein, The clock rate corresponds to the conversion rate of the digital-to-analog converter (DAC) of the corresponding audio playback chain.

7. The method according to any one of the preceding claims, wherein, Each audio channel signal includes multiple audio sub-channel signals processed by the same audio playback chain.

8. The method according to any one of the preceding claims, wherein, Each audio playback chain is either a local audio playback chain or a remote audio playback chain.

9. The method according to claim 8, wherein, The remote audio playback chain includes either a Bluetooth Low Energy audio playback chain or a WiFi audio playback chain.

10. The method according to claim 8 or 9, wherein, The delay signal of the corresponding audio playback chain indicates an audio playback delay. If the corresponding audio playback chain is a local audio playback chain, the audio playback delay includes the local presentation delay of the audio signal; or if the corresponding audio playback chain is a remote audio playback chain, the audio playback delay includes the transmission delay of the audio signal on the wireless medium and the remote presentation delay of the audio signal.

11. The method according to claim 10, wherein, The local presentation delay is the delay from when the audio signal enters the digital-to-analog converter (DAC) to when it is played back by the speaker, and the far-end presentation delay is the delay from when the audio signal is received by the far end of the far-end audio playback chain to be played back by the speaker.

12. The method according to any of the preceding claims, wherein, Estimating the sampling rate using the PLL based on the clock rate includes: smoothing the clock rate to estimate the sampling rate.

13. The method according to any one of the preceding claims, wherein, The corresponding audio channel signal is resampled based on the sampling rate to convert the source digital audio sampling rate of the audio channel signal to the sampling rate.

14. The method according to any one of the preceding claims, wherein, Generating the delayed resampled audio channel signals such that the two or more delayed resampled audio channel signals are synchronized in time during playback includes: delaying the resampled audio channel signals according to the difference between a preset delay and the delayed signals, wherein the preset delay is equal for all audio channel signals.

15. A method for estimating clock rate based on a delayed signal using a two-stage delay-locked loop (DLL), the method comprising: Receive the delayed signal, wherein the delayed signal corresponds to the signal delay on the signal transmission chain; Based on a first error signal between the delayed signal and a reference delay, a first clock rate is estimated using a first DLL level, wherein the first clock rate is estimated such that when the first clock rate is applied to a signal fed into the signal transmission chain, the delayed signal remains substantially constant over time. The pair of the first clock rate and the first error signal is accumulated over time until a threshold is reached; After the threshold is reached, the second clock rate corresponding to the first error signal equal to zero is estimated based on the accumulated pairs. Switch the delayed signal processing from the first DLL level to the second DLL level; and Based on the delayed signal and the second clock rate, the second DLL level is used to estimate the third clock rate, wherein the third clock rate is estimated such that when the third clock rate is applied to the signal fed to the signal transmission chain, the delayed signal remains substantially constant over time.

16. The method according to claim 15, wherein, Estimating the first clock rate using the first DLL level includes: A first averaging function is applied to the delayed signal to generate a first average delayed signal; Determine the first error signal between the first average delay signal and the reference delay; and The first clock rate is estimated by the proportional-derivative controller based on the error signal.

17. The method according to claim 16, wherein, The reference delay is the delay acquired during the startup of the signal transmission chain.

18. The method according to claim 16 or 17, wherein, The clock rate is estimated by the proportional-derivative controller based on a reference clock rate.

19. The method according to any one of claims 15 to 18, wherein, The threshold corresponds to the number of pairs of the accumulated first clock rate and the difference signal.

20. The method according to any one of claims 15 to 19, wherein, Estimating the second clock rate corresponding to the first error signal equal to zero based on accumulated pairs includes: Generate a histogram of the pair of the first clock rate and the first error signal; and The second clock rate is estimated based on the N pairs of the highest frequencies of the first clock rate and the first error signal in the histogram, where N is an integer greater than zero.

21. The method according to claim 20, wherein, The second clock rate is estimated by linear regression based on the N most frequent pairs of the first clock rate and the first error signal in the histogram.

22. The method according to any one of claims 15 to 21, wherein, Using the second DLL level to estimate the third clock rate includes: A second averaging function is applied to the delayed signal to generate a second averaged delayed signal; Determine a second error signal between the second average delay signal and a delayed version of the second average delay signal; and The proportional controller estimates the third clock rate based on the second error signal and the second clock rate.

23. The method according to claim 22, wherein when claim 22 is subordinate to any one of claims 16 to 18, wherein, The average window of the first averaging function is shorter than the average window of the second averaging function.

24. The method according to claim 23, wherein, The average window length of the first averaging function is equal to 1 second, and the average window length of the second averaging function is equal to 1 minute.

25. An apparatus comprising one or more processors and a memory, the memory being coupled to the one or more processors and storing instructions for the one or more processors, wherein, The one or more processors are adapted to perform the method according to any one of claims 1 to 24.

26. A computer program comprising instructions that, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 1 to 24.

27. A computer-readable storage medium storing a computer program according to claim 26.

Images