Methods, apparatus, and systems for heterogeneous audio playback synchronization

Abstract

Systems, methods, apparatus, programs, and storage media for synchronization of two or more audio channel signals processed by separate audio playback chains are described. The method is performed for each of the two or more audio channel signals. The method includes receiving a delay signal of the respective audio playback chain. A clock rate is estimated, using a first DLL, based on the delay signal. A sample rate is estimated, using a PLL, based on the clock rate. The respective audio channel signal is resampled based on the sample rate to generate a resampled audio channel signal. The resampled audio channel signal is delayed based on the delay signal to generate a delayed resampled audio channel signal such that the two or more delayed resampled audio channel signals are synchronized in time when played back.

Description

METHODS, APPARATUS, AND SYSTEMS FOR HETEROGENEOUS AUDIO PLAYBACK SYNCHRONIZATIONCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of priority from US Provisional Application Ser. No. 63 / 637,183, filed on 22 April 2024, and US Provisional Application Ser. No. 63 / 787,188 , filed on 11 April 2025, each of which is incorporated by reference herein in its entirety.TECHNICAL FIELD

[0002] The present disclosure relates to audio synchronization and more particularly to synchronization methods and devices for audio channels processed by different audio playback chains.BACKGROUND

[0003] Synchronizing audio channels processed by different playback chains is a challenging problem in audio engineering. A playback chain typically consists of a series of components — including digital signal processing (DSP) units, digital-to- analog converters (DACs), amplifiers, and speakers — each of which introduces its own processing time and potential variability. When multiple playback chains operate simultaneously, even small discrepancies in timing can lead to phase misalignment, comb filtering, unwanted echoes, or a breakdown of spatial imaging in multi-channel audio systems. These issues are especially pronounced in environments such as surround sound setups, live sound reinforcement, multi-room audio systems, and networked audio applications.

[0004] Several factors contribute to synchronization problems. Hardware variations, such as differences in DAC processing speeds or amplifier latency, can create timing mismatches between channels. In digital audio systems, differences in sample rate clocks between devices can cause drift, leading to gradual desynchronization over time. In networked audio environments, jitter and buffering delays — caused by variations in data transmission speeds — can further exacerbate the problem, making real-time synchronization difficult. Even in non-networkedscenarios, software-based processing (such as real-time DSP adjustments, equalization, and dynamic range compression) can introduce variable latency across different playback chains.

[0005] Thus, there is a need for a synchronization technique enabling the synchronous playback of audio channels that are processed by different audio playback chains.SUMMARY

[0006] In view of the above, the present disclosure provides methods, apparatus, and programs, as well as computer-readable storage media for synchronization of two or more audio channel signals processed by separate audio playback chains, and for estimating a clock rate based on a delay signal, having the features of the respective independent claims.

[0007] According to a first aspect of the disclosure, a method of synchronizing two or more audio channel signals, each audio channel signal being processed by a separate audio playback chain, is provided. The method may be performed for each audio channel signal. A delay signal of the respective audio playback chain may be received. A clock rate may be estimated / determined using a delay locked loop, DLL, based on the delay signal. The clock rate may be estimated such that the delay signal is kept substantially constant over time when the clock rate is applied to the audio channel signal. The clock rate may correspond or may be related to a conversion rate for a digital to analog converter, DAC of the respective audio playback chain. A sample rate may be estimated / determined using a phase locked loop, PLL, based on the clock rate. The audio channel signal may be resampled based on the sample rate to generate a resampled audio channel signal. The resampled audio channel signal may be delayed based on the delay signal to generate a delayed resampled audio channel signal such that the two or more delayed resampled audio channel signals are synchronized in time when played back.

[0008] By employing the synchronization method, two or more audio channel signals can be synchronized even if the corresponding audio playback chains introduce a completely different delay to the respective audio channel signal.

[0009] In some embodiments, the method may further include applying the clock rate to the delayed resampled audio channel signal to set a rate of transmitting samples of the delayed resampled audio channel signal over the respective audio playback chain.

[0010] In some embodiments, the setting a rate of transmitting the samples of the delayed resampled audio channel signal over the respective audio playback chain may include providing the samples to an audio interface of the respective audio playback chain (i.e., an interface for receiving a digital audio signal) at the set rate. The audio interface may for example be a host of a Bluetooth initiating device or a digital-to-analog converter, DAC.

[0011] In some embodiments, estimating, using the DLL, the clock rate based on the delay signal may include estimating, using a first DLL stage of the DLL, a first clock rate based on a first error signal between the delay signal and a reference delay. The first clock rate may be estimated such that the delay signal is kept substantially constant over time when the first clock rate is applied to the audio channel signal. Further, pairs of the first clock rate and the first error signal may be accumulated over time until a threshold is reached (e.g., a threshold related to a quantity of the pairs of the first clock rate and the first error signal). After the threshold is reached, a second clock rate may be estimated based on the accumulated pairs. The second clock rate may correspond to a first error signal being equal to zero. Further, the delay signal processing may be switched from the first DLL stage to a second DLL stage of the DLL (after the second clock rate has been estimated). Then, a third clock rate may be estimated / determined, using the second DLL stage based on the delay signal and the second clock rate. The third clock rate may be estimated such that the delay signal is kept substantially constant over time when the third clock rate is applied to the signal fed to the audio channel signal. The output clock rate may correspond to the first clock rate before switching from the first DLL stage to the second DLL stage, and my correspond to the third clock rate after switching from the first DLL stage to the second DLL stage.

[0012] By using the two-stage DLL, a stable clock rate may be estimated even if the input delay signal is coarsely quantized (e.g., due to inaccurate timers in the audio playback chain).

[0013] In some embodiments, each audio channel signal may include a plurality of audio subchannel signals processed by the same audio playback chain, i.e., multiple audio channel signals may use the same audio playback chain, or substantially the same audio playback chain, e.g., multiple local audio channel signals.

[0014] In some embodiments, each audio playback chain may be one of a local audio playback chain or a remote audio playback chain. The remote audio playback chain may include one of a Bluetooth low energy audio playback chain or a WiFi audio playback chain.

[0015] In some embodiments, the delay signal of the respective audio playback chain may be indicative of an audio playback delay that comprises a local presentation delay of the audio signal if the respective audio playback chain is a local audio playback chain, or that comprises a transportation delay of the audio signal over a wireless medium and a remote presentation delay of the audio signal if the respective audio playback chain is a remote audio playback chain. The local presentation delay may be a delay from the audio signal entering a DAC until being played back by a loudspeaker. The remote presentation delay may be a delay from the audio signal being received by a remote part of the remote audio playback chain until being played back by a loudspeaker.

[0016] In some embodiments, estimating the sample rate based on the clock rate using the PLL may include smoothing the clock rate to estimate / determine the sample rate.

[0017] In some embodiments, resampling the respective audio channel signal based on the sample rate may convert a source digital audio sample rate of the audio channel signal to the sample rate. The source digital audio sample rate may be a sample rate related to a codec used in the audio playback chain.

[0018] In some embodiments, generating the delayed resampled audio channel signal such that the two or more delayed resampled audio channel signals are synchronized in time when played back may include delaying the resampled audio channel signal according to a difference between a preset delay and the delay signal, wherein the preset delay is equal for all audio channel signals.

[0019] According to a second aspect of the disclosure, a two stage DLL based method of estimating a clock rate based on a delay signal is provided. A delay signal may be received. The delay signal may correspond to a signal delay on a signal transmission chain (i.e., signal transmission chain including digital processing). A first clock rate may be estimated, by using a first DLL stage, based on a first error signal between the delay signal and a reference delay. The first clock rate may be estimated such that the delay signal is kept substantially constant over time when the first clock rate is applied to a signal fed to the signal transmission chain. Further, pairs of the first clock rate and the first error signal may be accumulated over time until athreshold is reached (e.g., a threshold related to a quantity of the pairs of the first clock rate and the first error signal). After the threshold is reached, a second clock rate may be estimated based on the accumulated pairs. The second clock rate may correspond to a first error signal being equal to zero. Further, the delay signal processing may be switched from the first DLL stage to a second DLL stage of the DLL (after the second clock rate has been estimated). Then, a third clock rate may be estimated / determined, using the second DLL stage based on the delay signal and the second clock rate. The third clock rate may be estimated such that the delay signal is kept substantially constant over time when the third clock rate is applied to the signal fed to the signal transmission chain. The output clock rate may correspond to the first clock rate before switching from the first DLL stage to the second DLL stage, and my correspond to the third clock rate after switching from the first DLL stage to the second DLL stage.

[0020] By using the two stage DLL method, a first clock rate can be quickly estimated, while a stable third clock rate may be estimated after a certain quantity of first clock rate samples have been estimated. Thereby, a stable clock rate can be achieved, even if the delay signal input into the two stage DLL has large variations over time, e.g, due to a course quantization.

[0021] In some embodiments, estimating, using the first DLL stage, the first clock rate may comprise using a first averaging function on the delay signal to generate a first averaged delay signal. Further, the first error signal may be determined between the first averaged delay signal and the reference delay. The reference delay may be a delay acquired during startup of the signal transmission chain. The first clock rate may be estimated by a proportional derivate controller based on the error signal. Estimating the clock rate by the proportional derivate controller may be further based on a reference clock rate.

[0022] In some embodiments, estimating the second clock rate corresponding to the first error signal equal to zero based on the accumulated pairs may include generating a histogram of the pairs of the first clock rate and the first error signal. Further, the second clock rate may be estimated based on pairs of the first clock rate and the first error signal with the highest N frequencies in the histogram, where N is an integer larger than zero. Estimating the second clock rate based on the pairs of the first clock rate and the first error signal with the highest N frequencies in the histogram may performed by linear regression, i.e., linear regression may used to find a best fitting linear function to the pairs with the N highest frequencies, and the linear function may be used to determine a clock rate that corresponds to an error signal of zero.

[0023] In some embodiments, estimating, using the second DLL stage, the third clock rate may include using a second averaging function on the delay signal to generate a second averaged delay signal. Further, a second error signal may be determined between the second averaged delay signal and a delayed version of the second averaged delay signal. The third clock rate may be estimated / determined by a proportional controller based on the second error signal and the second clock rate.

[0024] In some embodiments, the first averaging function may have a shorter averaging window than the second averaging function. For example, a length of an averaging window of the first averaging function may be equal to 1 second, and a length of an averaging window of the second averaging function may be equal to 1 minute.

[0025] Any embodiment according to the second aspect may be combined with the two stage DLL embodiment according to the first aspect.

[0026] The first aspect and the second aspect of the present disclosure may be implemented via an apparatus. The apparatus may include a processor and memory coupled to the processor. The processor may be adapted carry out the method according to aspects and embodiments of the present disclosure.

[0027] Aspects of the present disclosure may be implemented via a program. When instructions of the program are executed by a processor, the processor may carry out aspects and embodiments of the present disclosure. A computer-readable storage medium may store the program. Such computer-readable storage media may include memory devices such as those described herein, including but not limited to random access memory (RAM) devices, read-only memory (ROM) devices, etc.. Accordingly, some innovative aspects of the subject matter described in this disclosure can be implemented via one or more computer-readable storage media having software stored thereon.

[0028] It will be appreciated that apparatus features and method steps may be interchanged in many ways. In particular, the details of the disclosed method(s) can be realized by the corresponding apparatus (or system), and vice versa, as the skilled person will appreciate. Moreover, any of the above statements made with respect to the method(s) are understood to likewise apply to the corresponding apparatus (or system), and vice versa.BRIEF DESCRIPTION OF DRAWINGS

[0029] Example embodiments of the disclosure are explained below with reference to the accompanying drawings, wherein

[0030] Fig. 1 schematically illustrates two different audio playback chains for two audio channel signals,

[0031] Fig. 2 schematically illustrates a Bluetooth LE Audio processing delay,

[0032] Fig. 3 schematically illustrates timings for a bus time of Bluetooth LE Audio,

[0033] Fig. 4 schematically illustrates an example with two synchronization subsystems for two audio channel signals according to embodiments of the disclosure,

[0034] Figs. 5 schematically illustrates an example of a two stage delay locked loop, DLL according to embodiments of the disclosure,

[0035] Fig. 6 schematically illustrates an example implementation of the two stage DLL according to embodiments of the disclosure,

[0036] Fig. 7 schematically illustrates an example histogram of error and clock rate samples and an estimation of the clock rate according to embodiments of the disclosure,

[0037] Fig. 8 is a flowchart illustrating an example of a method of synchronizing audio channel signals that are processed by different audio playback chains according to embodiments of the disclosure,

[0038] Fig. 9 is a flowchart illustrating an example of a method of estimating a clock rate by a two stage DLL according to embodiments of the disclosure, and

[0039] Fig. 10 schematically illustrates an example of an apparatus for synchronization of audio channels according to embodiments of the disclosure.DETAILED DESCRIPTION

[0040] The Figures (Figs.) and the following description relate to preferred embodiments by way of illustration only. It should be noted that from the following discussion, alternative embodiments of the structures and methods disclosed herein will be readily recognized as viable alternatives that may be employed without departing from the principles of what is claimed.

[0041] Reference will now be made in detail to several embodiments, examples of which are illustrated in the accompanying figures. It is noted that wherever practicable similar or like reference numbers may be used in the figures and may indicate similar or like functionality. The figures depict embodiments of the disclosed system (or method) for purposes of illustration only. One skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles described herein.Delays of audio playback chains

[0042] Fig. 1 illustrates a system for processing and synchronizing audio playback across multiple playback chains, including both local and remote audio playback paths.

[0043] The system comprises a software stack (SW Stack) that provides an audio signal with multiple audio channel signals. An audio channel signal is understood as an audio signal intended to be played back together with other audio channel signals of a multi-channel audio signal. The different audio channel signals may then be transmitted through different processing and playback paths.

[0044] In the example depicted in Fig. 1, the audio system comprises a local and a remote audio playback chain. In the local playback chain, the audio channel signal is routed directly to a DAC, followed by an amplifier, AMP, which ultimately drives a speaker. The delay introduced in this process is denoted as local presentation delay in the remaining description.

[0045] In the remote playback chain, the audio channel signal is transmitted wirelessly via Bluetooth (Bluetooth low energy, LE, audio), introducing an additional transportation delay. Upon reception, the signal may undergo additional processing, c.g., decoding, before being passed through a DAC and an AMP to reach the speaker. The total delay from the reception ofthe audio channel signal at the remote device until playback by the loudspeaker is denoted as remote presentation delay.

[0046] Fig. 1 highlights the potential synchronization issues caused by the transportation delay and the different presentation delays (remote and local), where the remote audio playback may lag behind the local playback. This issue may be particularly relevant in multi-speaker environments, such as wireless home theater systems, distributed audio setups, and Bluetooth- enabled sound synchronization solutions.

[0047] While Fig. 1 may depict two different playback chains for two audio channel signals, the disclosure shall not be construed to be limited to two audio channel signals. The synchronization method and system presented in the following description may be used in combination with an arbitrary number of audio channel signals (e.g., 7 channels) and an arbitrary number of channel signals may be assigned to local and remote playback chains (e.g., 2 audio channel signals may be assigned to local playback chains and 5 may be assigned to remote playback chains). Further, while Bluetooth LE audio is used as an example for the remote playback chain, the synchronization method and system in this application may be used with any wireless technology, suitable for transmitting audio signals, e.g., WIFI.

[0048] For some wireless technologies, a signal providing the total delay of the playback chain may not be available or may not be accurate enough. In these cases, additional timers for specific events in the playback chain may need to be added, such that a total delay of the respective playback chain can be determined.

[0049] An example determination of a total delay of a Bluetooth LE audio playback chain is provided in connection with Figs. 2 and 3.

[0050] Fig. 2 illustrates a timing sequence for the processing and transmission of an audio data unit, SDU within a Bluetooth LE Audio playback chain. The timeline depicts key delays introduced during the preparation and transport of the audio channel signal, highlighting multiple timestamps that may be necessary for determining the respective delays.

[0051] The sequence begins when the host device writes an SDU, initiating the data transfer process. The bus time to represents the delay associated with transferring the SDU from the hostto the controller. Following this, the preparation time accounts for internal processing (e.g., encoding) before transmission over the Bluetooth wireless channel.

[0052] Once the Bluetooth controller transmits the SDU, it references a synchronization event, SDU reference and introduces a Connected Isochronous Group, CIG sync delay, aligning the data transmission with a Bluetooth isochronous interval. The CIG sync event establishes a common timing reference for synchronized audio streaming across multiple receivers.

[0053] Subsequently, the data packet undergoes a flush timeout period, defined as (FT-1) x ISO_Interval, which determines the maximum allowable delay before the packet is discarded. Finally, the audio signal is processed by the remote Bluetooth device to be played back by a loudspeaker. The delay introduced by this type of processing may be the remote presentation delay depicted in Fig. 1.

[0054] Two delays, namely the bustime tO and the preperation time tl, may not be available or may not be determinable according to the current Bluetooth LE audio specification, i.e., Bluetooth 6.0.

[0055] For the preperation time tl, the current Bluetooth specification has a command HCI_LE_Read_ISO_TX_Sync to get a snapshot of the current SDU reference time. The reception time of the SDU and the Bluetooth controller may however not be available, and therefore the preparation time tl cannot be determined. To overcome this problem, a new timestamp is introduced in the Bluetooth controller.

[0056] Specifically, a new timestamp TreCv may be taken when the SDU arrived at the Bluetooth controller. Then, for each SDU, the preparation time may be calculated as Tp= TreCv - Tsdu, wherein Tsdu denotes the current SDU reference time. A HCl_Number_Of_Completed_Packets event may then be extended by providing Tpfor each SDU. An example extension is shown in Table 1 and Table 2.Table 1Where for each Connection_Handle|iJ, Preparation_Time[i][k] has the size of Num_Complctcd_Packcts[i] x 2 octets.Table 2

[0057] Further, Fig. 3 illustrates the bustime tO in the Bluetooth LE audio playback chain, representing the delay from when the host sends the SDU to when the Bluetooth controller receives it.

[0058] In modern operating systems, an application shall have T ' write readily available. The rest of the timestamps, such as T trigger (when the data transfer is starting), T_Cmpit (when the data transfer finished), and T_reCv (when the data is received by the Bluetooth controller) may not be as easy to obtain. The availability may depend on a specific drivcr / kcrncl implementation, run-time load of the system, and buffering between T_write and Trigger inside the kernel of the host.

[0059] In order to achieve a higher accuracy of the delay calculation, the host shall timestamp the SDU as close as possible to T_cmpit and provide this information to application. The Bluetooth controller shall timestamp the SDU packet as soon as the data transfer finishes such that time elapsed between T_cmpit and T_reCv may not be significant. The bustime tO may then be calculatated as T_c>npit - T write-

[0060] While an example for determining missing delays of a Bluetooth LE audio playback chains has been presented in connection with Figs. 2 and 3, and Tables 1 and 2, this example shall not be construed to limit the synchronization method and system according to embodiments of this application. It may be possible to determine different timestamps or determine respective delays in a different way. As long as complete delay can be communicated to the synchronization system, a synchronization of the different audio channel signals may be possible.

[0061] For the local presentation delay, some delays may be fixed during hardware design, while other delays may also be captured via timestamps. This may depend on the particular implementation of the local playback chain.Synchronization System

[0062] Next, the synchronization system will be described, which uses the determined delay to synchronize the different audio channel signals such that they are played back synchronously by the loudspeakers of the respective audio playback chains.

[0063] Fig. 4 schematically illustrated a synchronization system comprising two synchronization subsystems. The operation of each synchronization subsystem may be identical, irrespective of the particular audio playback chain, for which the synchronization subsystem is used. Therefore, the functionality of a synchronization subsystem will only be described with respect to the upper synchronization subsystem 100 in Fig. 4. It is again noted that the number of synchronization subsystems / audio playback chains and the type of the audio playback chains only serve as an example to illustrate the functionality of the synchronization system. Any number of audio playback chains and any types of audio playback chain may be used in combination with the synchronization system, as long as the quantity of synchronization subsystems equals the quantity of different audio playback chains. The synchronization system may be fully implemented in hardware, software or a mixture of both.

[0064] Synchronization subsystem 100 may comprise a delay locked loop, DLL 101. DLL 101 may receive a delay signal from the audio playback chain, e.g., from the Bluetooth LE audio playback chain. The delay signal may correspond to the total delay an audio channel signal experiences when transmitted through the audio playback chain, e.g., the total Bluetooth LE audio delay depicted in Fig. 2. The delay signal may be a time varying signal, i.e., a total delay may be reported at fixed intervals, e.g., in the order of milliseconds. The target of DLL 101 may be keeping the delay of the respective audio playback chain substantially constant over time. In particular, the total delay of the audio playback chain may depend on a clock signal used for driving the digital components (e.g., the DAC) of the audio playback chain. Therefore, the delay of the audio playback chain may be adjustable through clock rate of the clock signal. To control the delay of the audio playback chain, DLL 101 may determine a difference signal between the current value of the delay signal and a reference delay signal. The reference delay may be a reference delay of the respective audio playback chain. For example, the reference delay may be an initial total delay reported by the audio playback chain after startup. The difference between the current value of the delay signal and the reference delay may be denoted as an error signal. The error signal is fed into a controller of DLL 101. The controller of DLL 101 may be a functionthat determines a target clock rate for the digital part of the audio playback chain, such that the error signal is reduced when the target clock rate is applied to the audio playback chain. The controller may be any suitable controller to implement such a functionality, e.g. a proportional- integral-derivate, PID controller. The PID controller may be tuned according to known tuning methods.

[0065] The output of DLL 101 may therefore be a clock rate. Further, DLL 101 may also output the delay signal that has been received from the respective audio playback chain.

[0066] The clock rate may be the input of a phase-locked loop, PLL 102. PLL 102 may serve as a smoothing operation of the clock rate generated by DLL 101. In particular, PLL 102 functions as a feedback control system that continuously tracks, corrects, and stabilizes the input clock signal of DLL 101 by dynamically adjusting its output frequency. By using PLL 102, short-term fluctuations in the clock signal timing may be reduced. Any suitable PLL may be used as PLL 102. As the output signal of PLL 102 will be used to resample the audio channel signal, it will be referred to as sample rate in the following description.

[0067] Further, synchronization subsystem 100 may comprise resampler 103. Resampler 103 may receive the audio channel signal that is to be transmitted over the respective audio playback chain. The audio channel signal may be sampled according to a predefined frequency. The predefined frequency may for example depend on a codec used in the underlying technology of the respective audio playback chain. For example, Bluetooth LE audio may use the LC3 codec. Therefore, an audio channel signal intended for a Bluetooth LE playback audio chain may be sampled according to the LC3 codec.

[0068] The audio channel signal may comprise multiple audio subchannel signals. In other words, multiple audio subchannel signals may use the same type of audio playback chain.

[0069] Resampler 103 may receive the sample rate output by PLL 102 and may resample the audio channel signal from the original sample rate to the received sample rate, such that sample rate of the audio channel signal after resampler 103 matches the sample rate of the DAC of the respective audio playback chain. Thereby, a smooth operation of the DAC when converting the audio channel signal to the analog domain may be accomplished.

[0070] Resampler 103 therefore may output a resampled audio channel signal (resampled according to the sample rate determined by PLL 102) and may provide the resampled audio channel signal to delay line element 104.

[0071] Delay line element 104 may additionally receive the delay signal from DLL 101. The target of delay line element 104 is to introduce a delay for the resampled audio channel signal, such that the different audio channel signals are played back synchronously by the loudspeakers of the respective audio playback chains. This may be achieved in different ways. For example, delay line element 104 of each synchronization subsystem may have a same target delay and delays the signal such that the additional delay and the current value of the delay signal combined are equal to the target delay. Thereby, each audio channel signal may be delayed such that the experienced delay corresponds to the target delay.

[0072] Alternatively, the delay signal may be communicated between the different synchronization subsystems. Thereby, only the audio channel signals corresponding to the fast audio playback chains (e.g., the local audio playback chain) may need to be delayed. Therefore, a total delay of each audio playback chain may be dictated by the slowest audio playback chain.

[0073] Finally, DLL 101 may use the determined clock rate to determine a rate at which data of the delayed and resampled audio channel signal is fed to an interface of the respective audio playback chain. The type of interface may depend on the type of audio playback chain. For example, for a local audio playback chain, the interface may be the DAC of the audio playback chain. For a remote audio playback chain, the interface may be at the host of the system, e.g., the CPU of a desktop, a TV or a mobile phone or any other device that can initiate a Bluetooth connection.

[0074] In summary, by implementing a synchronization subsystem 100 for each audio playback chain, synchronous playback can be achieved for audio channel signals travelling through completely different types of audio playback chains.Improved DLL

[0075] Depending on a “quality” of the delay signal, a DLL such as DLL 104 may not be able to converge to a stable clock rate. In particular, the delay signal may be too coarse. For example, the delay signal may be quantized to a level that is too coarse for the DLL. This may happen whentime stamps are generated by a timer that has very low resolution in the audio playback chain.When a such a delay signal is the input to the DLL, the DLL may never be able to converge to a stable clock rate, but rather it may oscillate around the clock rate the DLL tries to match.

[0076] To enable a convergence to a clock rate for these circumstances, a two-stage DLL is proposed.

[0077] Fig. 5 illustrates an example of a two stage DLL 200. Two-stage DLL 200 may comprise a first DLL stage 210 and a second DLL stage 220. When DLL 200 is initialized, only the first DLL stage 210 may operate. In particular, first DLL stage 210 may receive the delay signal as described with respect to Fig. 4. Based on the delay signal and the reference delay, a first error signal is determined and fed to a controller of the first DLL stage 210 to estimate a first clock rate. As stated previously, first DLL stage 210 may not be able to converge to a stable clock rate.

[0078] In addition to outputting the first clock rate, DLL stage 210 may also transmit the first error signal and the first clock rate to a histogram accumulator 230. Histogram accumulator 230 may accumulate pairs of the first error signal and the first clock rate, i.c., pairs of the values of the first error signal and the first clock rate at a specific point in time. When a certain threshold is reached, the pairs of the first error signal and the first clock rate may be used to determine a histogram. The threshold may correspond to a quantity of pairs of the first error signal and the first clock rate. Alternatively, the threshold may be time based, i.e., histogram accumulator 230 may accumulate pairs of the first error signal and the first clock rate for a predefined time. When the threshold is reached, histogram accumulator 230 may determine a histogram based on the accumulated pairs of the first error signal and the first clock rate. The upper part of Fig. 7 illustrates an example for this histogram. In this example, rtmay denote the first clock rate and Ad, may denote the first error signal with i = [— K, K] denoting the bin index in the histogram. The number of bins and the bin size may be predefined or may depend on the actual values of the first error signal and the first clock rate. Further, the number of bins and the bin size may depend on the type of the audio playback chain.

[0079] Histogram accumulator 230 then may output the histogram to confidence based estimator 240. Confidence based estimator 240 may use the histogram to estimate a second clock rate, wherein the second clock rate may correspond to an error signal being equal to zero. Preferably, confidence based estimator 240 may only use the value pairs with the greatest frequency in thehistogram, to exclude outliers form the confidence based estimation of the second clock rate. In particular, confidence based estimator 240 may use the pairs of the first error signal and the first clock rate with the highest N frequencies, wherein N is an integer larger than zero. The estimation of the second clock rate may for example be performed by using linear regression. An example for estimating the second clock rate based on linear regression is illustrated in the lower pail of Fig. 7. Based on histogram accumulator 230 and confidence based estimator 240 a second clock rate corresponding to an error signal being zero can be estimated, even if the first clock rate output by first DLL stage 210 oscillates around this second clock rate.

[0080] After the second clock rate has been estimated, the first DLL stage 210 may stop its operation, and the second clock rate is transmitted to the second DLL stage 220.

[0081] The delay signal may now be the input of the second DLL stage 220, in order to estimate a second clock rate. DLL stage 220 may determine a second error signal, which corresponds to a difference between the delay signal and a delayed version of the delay signal. The second error signal may then be the input of a controller of the second DLL stage 220, which may also receive the second clock rate. By using the second error signal and the second clock rate as input, the controller of the second DLL stage 220 may be able to converge to a stable third clock rate. DLL stage 220 then may output the third clock rate together with the delay signal to synchronization subsystem 100.

[0082] Further, Fig. 6 illustrates an example implementation of the two-stage DLL 200. In this implementation example, the first DLL stage 210 may comprise window average block 211 and a proportional derivative, PD controller 212. Window average block 211 may perform an averaging function on the delay signal. In particular, the delay signal may be averaged over time by window average block 211 to generate a first averaged delay signal, and the first error signal may be determined based on the first averaged delay signal and the reference delay. PD controller 212 then determines / estimates the first clock rate based on the first error signal.Determining / estimating the first clock rate may be further based on a reference clock signal To.

[0083] The second DLL stage 220 may comprise window average block 221 and a proportional, P controller 222. Window average block 221 may also implement an averaging function, similar to window average block 211. The output of window average block 221 may be a second averaged delay signal. The averaging window of window average block 221 may however belonger than the averaging window of window average block 211. For example, the averaging window of window average block 221 may be 1 minute, while the averaging window of window average block 211 may be 1 second. The second error function may be determined based on the second averaged delay function and a delayed version of the second averaged delay function.

[0084] P controller 222 may estimate / determine the third clock rate based on the second error function and the second clock rate, which has been determined by confidence based estimator 240.

[0085] By using two- stage DLL 200, a stable clock rate can be estimated, even if the input delay signal is fluctuating too much, such that a stable clock rate cannot be estimated by single DLL stage.

[0086] In line with the above, a method 300 is provided for synchronization of two or more audio channel signals as depicted in the flowchart of Fig. 8. In addition to the following method steps, method 300 may optionally include all variations described above with respect to synchronization subsystem 100 and two-stage DLL 200 that have been described in connection with Fig. 4 to Fig. 7. Method 300 may be performed for each audio channel signal, wherein each audio channel signal is intended for a separate audio playback chain.

[0087] In step S301, a delay signal of a respective audio playback chain is received. The audio playback chain may be a local or a remote (wireless) audio playback chain. The delay signal may represent a total delay experienced by an audio channel signal when being processed by the audio playback chain, e.g., from an interface of the audio playback chain to a loudspeaker output of the audio playback chain.

[0088] In step S302, a clock rate is estimated / determined by a DLL based on the delay signal. The DLL may estimate / determine the clock rate such that the delay signal may be kept substantially constant when the clock rate is applied to the audio channel signal.

[0089] In step S3O3, a sample rate may be estimated / determined by a PLL based on the clock rate.

[0090] In step S304, the respective audio channel signal is resampled based on the sample rate to generate a resampled audio channel signal. An original sample rate of the audio channel signalmay be related to a codec used for the audio channel signal, while the estimated / determined sample rate may correspond to sample rate of a DAC in the respective audio playback chain.

[0091] In step S205, the resampled audio channel signal is delayed based on the delay signal to generate a delayed resampled audio channel signal such that the two or more delayed resampled audio channel signals are synchronized in time when played back. Synchronizing the delays may be based on a target delay.

[0092] Fig. 9 shows a method 400 for determining / estimating a clock rate by a two-stage DLL based on a delay signal. Method 400 may implement step S302, for example.

[0093] hi step 401, a delay signal is received. The delay signal corresponds to a signal delay on a signal transmission chain. The signal transmission chain may comprise digital components. For example, the signal transmission chain may be the audio playback chain in method 300.

[0094] In step 402, a first clock rate is estimated / determined by a first DLL stage based on a first error signal between the delay signal and a reference delay. The first clock rate is estimated such that the delay signal is kept substantially constant over time when the first clock rate is applied to a signal fed to the signal transmission chain. The first clock rate may be the output of method 400 after initialing the two-stage DLL.

[0095] In step 403, pairs of the first clock rate and the first error signal are accumulated over time until a threshold is reached. The threshold may correspond to a quantity of the pairs of the first clock rate and the first error signal that have been accumulated.

[0096] In step 404, a second clock rate is estimated based on the accumulated pairs, after the threshold is reached. The second clock rate corresponds to a first error signal equal to zero.

[0097] In step 405, the delay signal processing is switched from the first DLL stage to a second DLL stage. In other words, the first DLL stops operation and may no longer output the first clock rate and the second DLL may perform the clock rate estimation.

[0098] In step 406, a third clock rate is estimated / determined by the second LL based on the delay signal and the second clock rate. The third clock rate is estimated such that the delay signal is kept substantially constant over time when the third clock rate is applied to the signal fed to the signal transmission chain. The third clock rate may therefore be the output of method 400 after the switching operation in step 405.

[0099] While methods for synchronization of audio channel signals and for estimating a clock rate have been described above, the disclosure likewise relates to corresponding apparatus, and the like. An embodiment providing such apparatus will be described next with reference to Fig. 10.

[0100] As shown in Fig. 10, the apparatus 500 includes a processor 50 and memory 502. The memory 502 is configured to store program code. The processor 501 is configured to run instractions in the program code, so that the apparatus 500 performs synchronization of audio channel signals in any one of the above embodiments and implementations. The processor 501 may also receive, among others, suitable input data (e.g., audio channel signals and the delay signal), depending on use cases and / or implementations. The processor 501 may be adapted to carry out the methods / techniques (e.g., methods 300 and 400 as illustrated above with reference to Figs. 8 and 9, respectively) described throughout the present disclosure and to generate corresponding output data (e.g., the clock rate and the delayed resample audio cannel signal.), depending on use cases and / or implementations. The apparatus may be part of a device for orchestrating audio playback, e.g. a personal computer, a TV, or a smart speaker.

[0101] Aspects of the systems described herein may be implemented in an appropriate computer- based sound processing network environment for processing digital or digitized audio files. Portions of the adaptive audio system may include one or more networks that comprise any desired number of individual machines, including one or more routers (not shown) that serve to buffer and route the data transmitted among the computers. Such a network may be built on various different network protocols, and may be the Internet, a Wide Area Network (WAN), a Local Area Network (LAN), or any combination thereof.

[0102] One or more of the components, blocks, processes or other functional components may be implemented through a computer program that controls execution of a processor-based computing device of the system. It should also be noted that the various functions disclosed herein may be described using any number of combinations of hardware, firmware, and / or as data and / or instructions embodied in various machine-readable or computer-readable media, in terms of their behavioral, register transfer, logic component, and / or other characteristics. Computer- readable media in which such formatted data and / or instructions may be embodied include, but are not limited to, physical (non-transitory), non-volatile storage media in various forms, such as optical, magnetic or semiconductor storage media.

[0103] While one or more implementations have been described by way of example and in terms of the specific embodiments, it is to be understood that one or more implementations are not limited to the disclosed embodiments. To the contrary, it is intended to cover various modifications and similar arrangements as would be apparent to those skilled in the art. Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements.Interpretation

[0104] A computing device implementing the techniques described above can have the following example architecture. Other architectures are possible, including architectures with more or fewer components. In some implementations, the example architecture includes one or more processors (e.g., dual-core Intel® Xeon® Processors), one or more output devices (e.g., LCD), one or more network interfaces, one or more input devices (e.g., mouse, keyboard, touch- sensitive display) and one or more computer-readable mediums (e.g., RAM, ROM, SDRAM, hard disk, optical disk, flash memory, etc.). These components can exchange communications and data over one or more communication channels (e.g., buses), which can utilize various hardware and software for facilitating the transfer of data and control signals between components.

[0105] The term “computer-readable medium” refers to a medium that participates in providing instructions to processor for execution, including without limitation, non-volatile media (e.g., optical or magnetic disks), volatile media (e.g., memory) and transmission media. Transmission media includes, without limitation, coaxial cables, copper wire and fiber optics.

[0106] Computer-readable medium can further include operating system (e.g., a Linux® operating system), network communication module, audio interface manager, audio processing manager and live content distributor. Operating system can be multi-user, multiprocessing, multitasking, multithreading, real time, etc. Operating system performs basic tasks, including but not limited to: recognizing input from and providing output to network interfaces and / or devices; keeping track and managing files and directories on computer-readable mediums (e.g., memory or a storage device); controlling peripheral devices; and managing traffic on the one or more communication channels. Network communications module includes various components for establishing and maintaining network connections (e.g., software for implementing communication protocols, such as TCP / IP, HTTP, etc.).

[0107] Architecture can be implemented in a parallel processing or peer-to-peer infrastructure or on a single device with one or more processors. Software can include multiple software components or can be a single body of code.

[0108] The described features can be implemented advantageously in one or more computer programs that are executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. A computer program is a set of instructions that can be used, directly or indirectly, in a computer to perform a certain activity or bring about a certain result. A computer program can be written in any form of programming language (e.g., Objective-C, Java), including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, a browser-based web application, or other unit suitable for use in a computing environment.

[0109] Suitable processors for the execution of a program of instructions include, by way of example, both general and special purpose microprocessors, and the sole processor or one of multiple processors or cores, of any kind of computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memories for storing instructions and data. Generally, a computer will also include, or be operatively coupled to communicate with, one or more mass storage devices for storing data files; such devices include magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and optical disks. Storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, ASICs (application-specific integrated circuits).

[0110] To provide for interaction with a user, the features can be implemented on a computer having a display device such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor or a retina display device for displaying information to the user. The computer can have a touch surface input device (e.g., a touch screen) or a keyboard and a pointing device such as a mouse ora trackball by which the user can provide input to the computer. The computer can have a voice input device for receiving voice commands from the user.

[0111] The features can be implemented in a computer system that includes a back-end component, such as a data server, or that includes a middleware component, such as an application server or an Internet server, or that includes a front-end component, such as a client computer having a graphical user interface or an Internet browser, or any combination of them. The components of the system can be connected by any form or medium of digital data communication such as a communication network. Examples of communication networks include, e.g., a LAN, a WAN, and the computers and networks forming the Internet.

[0112] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server transmits data (e.g., an HTML page) to a client device (e.g., for purposes of displaying data to and receiving user input from a user interacting with the client device). Data generated at the client device (e.g., a result of the user interaction) can be received from the client device at the server.

[0113] A system of one or more computers can be configured to perform particular actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions. One or more computer programs can be configured to perform particular actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions.

[0114] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any inventions or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or morefeatures from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0115] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0116] Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that throughout the present invention discussions utilizing terms such as “processing”, “computing”, “calculating”, “determining”, “analyzing” or the like, refer to the action and / or processes of a computer or computing system, or similar electronic computing devices, that manipulate and / or transform data represented as physical, such as electronic, quantities into other data similarly represented as physical quantities.

[0117] Reference throughout this invention to “one example embodiment”, “some example embodiments” or “an example embodiment” means that a particular feature, structure or characteristic described in connection with the example embodiment is included in at least one example embodiment of the present invention. Thus, appearances of the phrases “in one example embodiment”, “in some example embodiments” or “in an example embodiment” in various places throughout this invention are not necessarily all referring to the same example embodiment. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner, as would be apparent to one of ordinary skill in the art from this invention, in one or more example embodiments.

[0118] As used herein, unless otherwise specified the use of the ordinal adjectives “first”, “second”, “third”, etc., to describe a common object, merely indicate that different instances of like objects are being referred to and are not intended to imply that the objects so described must be in a given sequence, either temporally, spatially, in ranking, or in any other manner.

[0119] Also, it is to be understood that the phraseology and terminology used herein are for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” or “having” and variations thereof are meant to encompass the items listed thereafter and equivalents thereof as well as additional items. Unless specified or limited otherwise, the terms “mounted”, “connected”, “supported”, and “coupled” and variations thereof are used broadly and encompass both direct and indirect mountings, connections, supports, and couplings.

[0120] In the claims below and the description herein, any one of the terms comprising, comprised of or which comprises is an open term that means including at least the elements / features that follow, but not excluding others. Thus, the term comprising, when used in the claims, should not be interpreted as being limitative to the means or elements or steps listed thereafter. For example, the scope of the expression a device comprising A and B should not be limited to devices consisting only of elements A and B. Any one of the terms including or which includes or that includes as used herein is also an open term that also means including at least the elements / features that follow the term, but not excluding others. Thus, including is synonymous with and means comprising.

[0121] It should be appreciated that in the above description of example embodiments of the present invention, various features of the present invention are sometimes grouped together in a single example embodiment, Fig., or description thereof for the purpose of streamlining the present invention and aiding in the understanding of one or more of the various inventive aspects. This method of invention, however, is not to be interpreted as reflecting an intention that the claims require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed example embodiment. Thus, the claims following the Description are hereby expressly incorporated into this Description, with each claim standing on its own as a separate example embodiment of this invention.

[0122] Furthermore, while some example embodiments described herein include some but not other features included in other example embodiments, combinations of features of different example embodiments are meant to be within the scope of the present invention, and form different example embodiments, as would be understood by those skilled in the art. For example,in the following claims, any of the claimed example embodiments can be used in any combination.

[0123] In the description provided herein, numerous specific details are set forth. However, it is understood that example embodiments of the present invention may be practiced without these specific details. In other instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

[0124] Thus, while there has been described what are believed to be the best modes of the present invention, those skilled in the art will recognize that other and further modifications may be made thereto without departing from the spirit of the present invention, and it is intended to claim all such changes and modifications as fall within the scope of the present invention. For example, any formulas given above arc merely representative of procedures that may be used. Functionality may be added or deleted from the block diagrams and operations may be interchanged among functional blocks. Steps may be added or deleted to methods described within the scope of the present disclosure.

[0125] Various aspects and implementations of the present disclosure may also be appreciated from the following enumerated example embodiments (EEEs), which are not claims.EEE 1. A method of synchronizing two or more audio channel signals, each audio channel signal being processed by a separate audio playback chain, the method comprising, for each audio channel signal: receiving a delay signal of the respective audio playback chain; estimating, using a delay locked loop, DLL, a clock rate based on the delay signal; estimating, using a phase locked loop, PLL, a sample rate based on the clock rate; resampling the respective audio channel signal based on the sample rate to generate a resampled audio channel signal; and delaying the resampled audio channel signal based on the delay signal to generate a delayed resampled audio channel signal such that the two or more delayed resampled audio channel signals are synchronized in time when played back.EEE 2. The method according to EEE 1, wherein the clock rate is estimated such that the delay signal is kept substantially constant over time when the clock rate is applied to the audio channel signal.EEE 3. The method according to EEE 1 or 2, wherein the method further comprises: applying the clock rate to the delayed resampled audio channel signal to set a rate of transmitting samples of the delayed resampled audio channel signal over the respective audio playback chain.EEE 4. The method according to EEE 3, wherein setting a rate of transmitting the samples of the delayed resampled audio channel signal over the respective audio playback chain comprises providing the samples to an audio interface of the respective audio playback chain at the set rate.EEE 5. The method according to any previous EEE, wherein estimating, using the DLL, the clock rate based on the delay signal comprises: estimating, using a first DLL stage of the DLL, a first clock rate based on a first error signal between the delay signal and a reference delay, and wherein the first clock rate is estimated such that the delay signal is kept substantially constant over time when the first clock rate is applied to the audio channel signal; accumulating pairs of the first clock rate and the first error signal over time until a threshold is reached; after the threshold is reached, estimating a second clock rate corresponding to a first error signal equal to zero based on the accumulated pairs; switching the delay signal processing from the first DLL stage to a second DLL stage of the DLL; and estimating, using the second DLL stage, a third clock rate based on the delay signal and the second clock rate, wherein the third clock rate is estimated such that the delay signal is kept substantially constant over time when the third clock rate is applied to the signal fed to the audio channel signal; wherein before switching the delay signal processing from the first DLL stage to the second DLL stage, the clock rate corresponds to the first clock rate and after switching the delaysignal processing from the first DLL stage to the second DLL stage, the clock rate corresponds to the third clock rate.EEE 6. The method according to any previous EEE, wherein the clock rate corresponds to a conversion rate for a digital to analog converter, DAC of the respective audio playback chain.EEE 7. The method according to any previous EEE, wherein each audio channel signal comprises a plurality of audio subchannel signals processed by the same audio playback chain.EEE 8. The method according to any previous EEE, wherein each audio playback chain is one of a local audio playback chain or a remote audio playback chain.EEE 9. The method according to EEE 8, wherein the remote audio playback chain comprises one of a Bluetooth low energy audio playback chain or a WiFi audio playback chain.EEE 10. The method according to EEE 8 or 9, wherein the delay signal of the respective audio playback chain is indicative of an audio playback delay that comprises a local presentation delay of the audio signal if the respective audio playback chain is a local audio playback chain, or that comprises a transportation delay of the audio signal over a wireless medium and a remote presentation delay of the audio signal if the respective audio playback chain is a remote audio playback chain.EEE 11. The method according to EEE 10, wherein the local presentation delay is a delay from the audio signal entering a digital to analog converter, DAC until being played back by a loudspeaker, and wherein the remote presentation delay is a delay from the audio signal being received by a remote part of the remote audio playback chain until being played back by a loudspeaker.EEE 12. The method according to any previous EEE, wherein estimating the sample rate based on the clock rate using the PLL comprises smoothing the clock rate to estimate the sample rate.EEE 13. The method according to any previous EEE, wherein resampling the respective audio channel signal based on the sample rate converts a source digital audio sample rate of the audio channel signal to the sample rate.EEE 14. The method according to any previous claim, wherein generating the delayed resampled audio channel signal such that the two or more delayed resampled audio channel signals are synchronized in time when played back comprises delaying the resampled audio channel signal according to a difference between a preset delay and the delay signal, wherein the preset delay is equal for all audio channel signals.EEE 15. A two stage delay locked loop, DLL based method of estimating a clock rate based on a delay signal, the method comprising: receiving the delay signal, wherein the delay signal corresponds to a signal delay on a signal transmission chain; estimating, using a first DLL stage, a first clock rate based on a first error signal between the delay signal and a reference delay, and wherein the first clock rate is estimated such that the delay signal is kept substantially constant over time when the first clock rate is applied to a signal fed to the signal transmission chain; accumulating pairs of the first clock rate and the first error signal over time until a threshold is reached; after the threshold is reached, estimating a second clock rate corresponding to a first error signal equal to zero based on the accumulated pairs; switching the delay signal processing from the first DLL stage to a second DLL stage; and estimating, using the second DLL stage, a third clock rate based on the delay signal and the second clock rate, wherein the third clock rate is estimated such that the delay signal is kept substantially constant over time when the third clock rate is applied to the signal fed to the signal transmission chain.EEE 16. The method according to EEE 15, wherein estimating, using the first DLL stage, the first clock rate comprises:using a first averaging function on the delay signal to generate a first averaged delay signal; determining the first error signal between the first averaged delay signal and the reference delay; and estimating the first clock rate by a proportional derivate controller based on the error signal.EEE 17. The method according to EEE 16, wherein the reference delay is a delay acquired during startup of the signal transmission chain.EEE 18. The method according to EEE 16 or 17, wherein estimating the clock rate by the proportional derivate controller is further based on a reference clock rate.EEE 19. The method according to any one of EEEs 15 to 18, wherein the threshold corresponds to a quantity of accumulated pairs of the first clock rate and the difference signal.EEE 20. The method according to any one of EEEs 15 to 19, wherein estimating the second clock rate corresponding to the first error signal equal to zero based on the accumulated pairs comprises: generating a histogram of the pairs of the first clock rate and the first error signal; and estimating the second clock rate based on pairs of the first clock rate and the first error signal with the highest N frequencies in the histogram, where N is an integer larger than zero.EEE 21. The method according to EEE 20, wherein estimating the second clock rate based on the pairs of the first clock rate and the first error signal with the highest N frequencies in the histogram is performed by linear regression.EEE 22. The method according to any one of EEEs 15 to 21, wherein estimating, using the second DLL stage, the third clock rate comprises: using a second averaging function on the delay signal to generate a second averaged delay signal;determining a second error signal between the second averaged delay signal and a delayed version of the second averaged delay signal; and estimating the third clock rate by a proportional controller based on the second error signal and the second clock rate.EEE 23. The method according to EEE 22 when depending on any one of EEEs 16 to 18, wherein the first averaging function has a shorter averaging window than the second averaging function.EEE 24. The method according to EEE 23, wherein a length of an averaging window of the first averaging function is equal to 1 second, and a length of an averaging window of the second averaging function is equal to 1 minute.EEE 25. An apparatus, comprising one or more processors and a memory coupled to the one or more processors and storing instructions for the one or more processors, wherein the one or more processors arc adapted to carry out the method according to any one of EEEs 1 to 24.EEE 26. A computer program comprising instructions that, when executed by one or more processors, cause the one or more processors to carry out the method according to any one of EEEs 1 to 24.EEE 27. A computer-readable storage medium storing the computer program according to EEE 26.EEE 28. A system for synchronously playing back a first audio channel transmitted through a first audio codec and a second audio channel transmitted through a second audio codec, the system comprising: a first channel module comprising: a first channel resampler configured to convert a source digital audio sampling rate of the first audio channel to a first digital-to-analog conversion (DAC) sampling rate; a first channel phase locked loop configured to provide the first DAC sampling rate to the first channel resampler;a first channel delay locked loop configured to: synchronize the first DAC sampling rate with characteristics of the first audio codec; and maintain a first delay for the first audio channel at a first transmission reference point; a first channel delay line configured to: provide the first audio channel with the first DAC sampling rate and first delay to a first audio interface; and a second channel module comprising: a second channel resampler configured to convert a source digital audio sampling rate of the second audio channel to a second digital-to-analog conversion (DAC) sampling rate; a second channel phase locked loop configured to provide the second DAC sampling rate to the second channel resampler; a second channel delay locked loop configured to: synchronize the second DAC sampling rate with characteristics of the second audio codec; and maintain a second delay for the second audio channel at a second transmission reference point; a second channel delay line configured to: provide the second audio channel with the second DAC sampling rate and second delay to a second audio interface.EEE 29. The system of EEE 28, further comprising: a plurality of first audio channels, wherein the plurality of first audio channels comprises at least the first audio channel; and a plurality of second audio channels, wherein the plurality of second audio channels comprises at least the second audio channel.EEE 30. The system of EEE 28 or 29, wherein the first audio codec comprises at least one of Bluetooth LE Audio codec or a local audio codec and the second audio codec comprises at least one of the Bluetooth LE Audio codec or the local audio codec.EEE 31. The system of any one of EEEs 28-30, wherein the characteristics of the first audio codec comprise a latency associated with the first audio codec, and wherein thecharacteristics of the second audio codec comprise a latency associated with the second audio codec.EEE 32. The system of any one of EEEs 28-31, wherein the first delay or the second delay comprises at least one of a local presentation delay, a remote presentation delay, and / or a transportation delay.EEE 33. The system of any one of EEEs 28-32, wherein the first audio interface or the second audio interface comprises at least one of a Bluetooth LE audio interface and / or a local audio interface.EEE 34. A delay locked loop (DLL) system for determining a conversion rate and delay, the system comprising: an estimation stage configured to: estimate a conversion clock rate sample; maintain a constant delay; record, in a histogram, an error of the constant delay and the conversion clock rate sample; perform linear regression based on the histogram to estimate one or more initial conversion clock rate samples; a control stage: determine, from the one or more estimated initial conversion clock rate samples, the conversion rate and the delay.EEE 35. The delay locked loop system of EEE 34, wherein the estimation stage comprises a proportional-derivation controller.EEE 36. The delay locked loop system of EEE 34 or 35, wherein the control stage comprises a proportional controller.EEE 37. The delay locked loop system of any one of EEEs 34-37, wherein data points in the histogram with highest N frequencies are used to perform the linear regression.EEE 38. The delay locked loop system of EEE 37, wherein the data points comprise the error of the constant delay and / or the conversion clock rate sample.

Claims

CLAIMS1. A method of synchronizing two or more audio channel signals, each audio channel signal being processed by a separate audio playback chain, the method comprising, for each audio channel signal: receiving a delay signal of the respective audio playback chain; estimating, using a delay locked loop, DLL, a clock rate based on the delay signal; estimating, using a phase locked loop, PLL, a sample rate based on the clock rate; resampling the respective audio channel signal based on the sample rate to generate a resampled audio channel signal; and delaying the resampled audio channel signal based on the delay signal to generate a delayed resampled audio channel signal such that the two or more delayed resampled audio channel signals are synchronized in time when played back.

2. The method according to claim 1, wherein the clock rate is estimated such that the delay signal is kept substantially constant over time when the clock rate is applied to the audio channel signal.

3. The method according to claim 1 or 2, wherein the method further comprises: applying the clock rate to the delayed resampled audio channel signal to set a rate of transmitting samples of the delayed resampled audio channel signal over the respective audio playback chain.

4. The method according to claim 3, wherein setting a rate of transmitting the samples of the delayed resampled audio channel signal over the respective audio playback chain comprises providing the samples to an audio interface of the respective audio playback chain at the set rate.

5. The method according to any previous claim, wherein estimating, using the DLL, the clock rate based on the delay signal comprises: estimating, using a first DLL stage of the DLL, a first clock rate based on a first error signal between the delay signal and a reference delay, and wherein the first clock rate is estimatedsuch that the delay signal is kept substantially constant over time when the first clock rate is applied to the audio channel signal; accumulating pairs of the first clock rate and the first error signal over time until a threshold is reached; after the threshold is reached, estimating a second clock rate corresponding to a first error signal equal to zero based on the accumulated pairs; switching the delay signal processing from the first DLL stage to a second DLL stage of the DLL; and estimating, using the second DLL stage, a third clock rate based on the delay signal and the second clock rate, wherein the third clock rate is estimated such that the delay signal is kept substantially constant over time when the third clock rate is applied to the signal fed to the audio channel signal; wherein before switching the delay signal processing from the first DLL stage to the second DLL stage, the clock rate corresponds to the first clock rate and after switching the delay signal processing from the first DLL stage to the second DLL stage, the clock rate corresponds to the third clock rate.

6. The method according to any previous claim, wherein the clock rate corresponds to a conversion rate for a digital to analog converter, DAC of the respective audio playback chain.

7. The method according to any previous claim, wherein each audio channel signal comprises a plurality of audio subchannel signals processed by the same audio playback chain.

8. The method according to any previous claim, wherein each audio playback chain is one of a local audio playback chain or a remote audio playback chain.

9. The method according to claim 8, wherein the remote audio playback chain comprises one of a Bluetooth low energy audio playback chain or a WiFi audio playback chain.

10. The method according to claim 8 or 9, wherein the delay signal of the respective audio playback chain is indicative of an audio playback delay that comprises a local presentation delay of the audio signal if the respective audio playback chain is a local audio playback chain, or thatcomprises a transportation delay of the audio signal over a wireless medium and a remote presentation delay of the audio signal if the respective audio playback chain is a remote audio playback chain.

11. The method according to claim 10, wherein the local presentation delay is a delay from the audio signal entering a digital to analog converter, DAC until being played back by a loudspeaker, and wherein the remote presentation delay is a delay from the audio signal being received by a remote part of the remote audio playback chain until being played back by a loudspeaker.

12. The method according to any previous claim, wherein estimating the sample rate based on the clock rate using the PLL comprises smoothing the clock rate to estimate the sample rate.

13. The method according to any previous claim, wherein resampling the respective audio channel signal based on the sample rate converts a source digital audio sample rate of the audio channel signal to the sample rate.

14. The method according to any previous claim, wherein generating the delayed resampled audio channel signal such that the two or more delayed resampled audio channel signals are synchronized in time when played back comprises delaying the resampled audio channel signal according to a difference between a preset delay and the delay signal, wherein the preset delay is equal for all audio channel signals.

15. A two stage delay locked loop, DLL based method of estimating a clock rate based on a delay signal, the method comprising: receiving the delay signal, wherein the delay signal corresponds to a signal delay on a signal transmission chain; estimating, using a first DLL stage, a first clock rate based on a first error signal between the delay signal and a reference delay, and wherein the first clock rate is estimated such that the delay signal is kept substantially constant over time when the first clock rate is applied to a signal fed to the signal transmission chain;accumulating pairs of the first clock rate and the first error signal over time until a threshold is reached; after the threshold is reached, estimating a second clock rate corresponding to a first error signal equal to zero based on the accumulated pairs; switching the delay signal processing from the first DLL stage to a second DLL stage; and estimating, using the second DLL stage, a third clock rate based on the delay signal and the second clock rate, wherein the third clock rate is estimated such that the delay signal is kept substantially constant over time when the third clock rate is applied to the signal fed to the signal transmission chain.

16. The method according to claim 15, wherein estimating, using the first DLL stage, the first clock rate comprises: using a first averaging function on the delay signal to generate a first averaged delay signal; determining the first error signal between the first averaged delay signal and the reference delay; and estimating the first clock rate by a proportional derivate controller based on the error signal.

17. The method according to claim 16, wherein the reference delay is a delay acquired during startup of the signal transmission chain.

18. The method according to claim 16 or 17, wherein estimating the clock rate by the proportional derivate controller is further based on a reference clock rate.

19. The method according to any one of claims 15 to 18, wherein the threshold corresponds to a quantity of accumulated pairs of the first clock rate and the difference signal.

20. The method according to any one of claims 15 to 19, wherein estimating the second clock rate corresponding to the first error signal equal to zero based on the accumulated pairs comprises:generating a histogram of the pairs of the first clock rate and the first error signal; and estimating the second clock rate based on pairs of the first clock rate and the first error signal with the highest N frequencies in the histogram, where N is an integer larger than zero.

21. The method according to claim 20, wherein estimating the second clock rate based on the pairs of the first clock rate and the first error signal with the highest N frequencies in the histogram is performed by linear regression.

22. The method according to any one of claims 15 to 21, wherein estimating, using the second DLL stage, the third clock rate comprises: using a second averaging function on the delay signal to generate a second averaged delay signal; determining a second error signal between the second averaged delay signal and a delayed version of the second averaged delay signal; and estimating the third clock rate by a proportional controller based on the second error signal and the second clock rate.

23. The method according to claim 22 when depending on any one of claims 16 to 18, wherein the first averaging function has a shorter averaging window than the second averaging function.

24. The method according to claim 23, wherein a length of an averaging window of the first averaging function is equal to 1 second, and a length of an averaging window of the second averaging function is equal to 1 minute.

25. An apparatus, comprising one or more processors and a memory coupled to the one or more processors and storing instructions for the one or more processors, wherein the one or more processors are adapted to carry out the method according to any one of claims 1 to 24.

26. A computer program comprising instructions that, when executed by one or more processors, cause the one or more processors to carry out the method according to any one of claims 1 to 24.

27. A computer-readable storage medium storing the computer program according to claim 26.

Images