Clock synchronization using wireless side channels

Abstract

Individual clock adjustments between electronic devices are typically based on round trip time (RTT) measurements of reference messages between an originating device and a receiving device. With increasing expectations for clock synchronization accuracy and widespread use of wireless data networks, the technology disclosed herein provides a dedicated clock synchronization network that generates fixed delays between each hop and within associated devices of the dedicated clock synchronization network. By taking into account known delays between each hop and within associated devices of the dedicated clock synchronization network, better clock synchronization accuracy can be achieved than prior art techniques that estimate latency based on RTT measurements.

Description

Background Art

[0001] Clock synchronization is used to coordinate otherwise independent clocks in electronic devices. Even if initially set accurately, independent clocks can vary over time due to clock drift, which is caused by the independent clocks ticking at slightly different rates. Clock discrepancies in electronic devices lead to a variety of problems and a variety of solutions (clock synchronization techniques), each with their own limitations.

[0002] For example, in managing time at smaller and smaller scales, the problems associated with clock drift become more complex in distributed computing, where several computers need to achieve the same global time to work properly. In addition, clock synchronization is important for the accurate reproduction of streaming media, especially when using multiple input and output devices.

[0003] In a system with a central control server, the server can specify the system time in a hub-and-spoke arrangement. The Cristian algorithm and the Berkeley algorithm can be used to resolve clock drift in this environment and achieve clock synchronization. In a distributed system, clock synchronization is more complicated because it is not easy to know the single controlling global time.

[0004] The most commonly used clock synchronization technology on the Internet is the Network Time Protocol (NTP), which is a hierarchical client-server architecture based on User Datagram Protocol (UDP) messaging and its round-trip time (RTT) measurement. NTP can reduce synchronization offsets to the order of a few milliseconds on the public Internet and to sub-millisecond levels on local area networks. Simple Network Time Protocol (SNTP) and Precision Time Protocol (PTP) are master / slave synchronization protocols used on local area networks and are similarly based on RTT measurements of UDP messages or other data packets between master and slave devices. Summary of the invention

[0005] The implementations described and claimed herein provide a clock synchronization network including a master device and a slave device. The master device includes a master timing circuit system for generating a master timing reference and a data output for transmitting the master timing reference on a dedicated data channel. The slave device includes a data input for receiving the master timing reference on the dedicated data channel and a slave timing circuit system for synchronizing the slave timing reference to the master timing reference using a constant delay on the dedicated data channel.

[0006] Implementations described and claimed herein also provide a method for synchronizing a slave device to a master device, comprising: generating a master timing reference using a master timing circuit system, transmitting the master timing reference outwardly from the master device via a dedicated data channel, receiving the master timing reference having a constant delay from the master device to the slave device on the dedicated data channel, and synchronizing the slave timing reference of the slave device to the master timing reference using the constant delay on the dedicated data channel.

[0007] Other implementations are also described and listed herein. This Summary is provided to introduce a selection of concepts that are further described below in the Detailed Description in a simplified form. This Disclosure is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 An example dedicated wireless clock synchronization network for synchronizing slave devices to a timing reference broadcast from a master device is shown.

[0009] Figure 2 An example synchronization timeline is shown using timestamps to synchronize slave devices to a timing reference broadcast from a master device.

[0010] Figure 3 Another example synchronization timeline is shown that synchronizes slave devices to a timing reference that is periodically broadcast from a master device at a known packet rate.

[0011] Figure 4 A dedicated wireless clock synchronization network of slave speakers synchronized to a master speaker is shown.

[0012] Figure 5 A dedicated wireless clock synchronization network of slave microphones synchronized to a master microphone is shown.

[0013] Figure 6 A dedicated wireless clock synchronization network of slave cameras synchronized to a master camera is shown.

[0014] Figure 7 An example operation of synchronizing a slave device to a master device is shown.

[0015] Figure 8 Example hardware and software that may be useful in implementing the described techniques are shown. DETAILED DESCRIPTION

[0016] In wireless networks, managing clock drift becomes more challenging due to the collision potential of synchronization packets over the wireless medium and the higher clock drift rates on low-cost wireless devices. Reference Broadcast Time Synchronization (RBS) algorithms are used in wireless networks and sensor networks, where an initiating device broadcasts a reference message to prompt receiving devices to adjust their clocks. However, individual clock adjustments are still based on RTT measurements of the reference message between the initiating device and the receiving device.

[0017] Because RTT varies over time, especially over the Internet or using shared device hardware and / or software, existing clock synchronization techniques based on RTT measurements have difficulty achieving extremely high accuracy (e.g., within one time tick (the resolution of the clock source), which may be + / - 1 microsecond in an implementation using a 1Mhz clock source).

[0018] With the increasing expectations for clock synchronization accuracy and the widespread use of wireless data networks, the technology disclosed herein provides a dedicated clock synchronization network that generates fixed delays between each hop and within the associated devices of the dedicated clock synchronization network. By considering the known delays between each hop and within the associated devices of the dedicated clock synchronization network, better clock synchronization accuracy can be achieved than the existing technology that estimates latency based on RTT measurements. In addition, synchronization can be achieved in a wireless ad hoc network by sending synchronization messages in a multi-hop manner, and each subsequent node uses an additive delay based on the total number of hops between the nodes to gradually synchronize with the direct sender node of the synchronization message.

[0019] Master / Slave is an asymmetric communication or control model where one device (the master) controls one or more other devices (the slaves) and acts as their communication hub. The master described herein provides master timing to the slaves. The slaves described herein receive the master timing reference and adjust their corresponding slave timing reference to match.

[0020] Various computing devices (e.g., laptops, personal computers, gaming devices, smartphones, smart televisions, wireless speakers, projectors and microphones, and other devices that perform one or more sets of specific arithmetic and / or logical operations) can utilize the clock synchronization networks and signals disclosed in detail herein.

[0021] Figure 1 An example dedicated wireless clock synchronization network 102 is shown for synchronizing slave devices to a master timing reference 106 broadcast from a master device 108. The master device 108 includes an electrical signal X that generates a precisely known frequency. m The master timing element of the oscillator (e.g., a crystal oscillator 110 or an electronic oscillator such as a microcontroller and a phase-locked loop (PLL) circuit). mis input into a master control unit 112 (MCU, also referred to herein as a master timing circuit system), which includes a software configurable timer or phase-locked loop, for example, to generate a master timing reference 106. The master timing reference 106 is output to various components or peripherals of the master device 108 for coordinated operation of the master device 108.

[0022] The slave device 104 (and other slave devices connected to the network 102) includes a device that generates an electrical signal X s The electrical signal X s Similar to the electrical signal X m , but it is generated by the slave crystal oscillator 116 independently of the master crystal oscillator 110. s is input into a slave control unit 118 (SCU, also referred to herein as a slave timing circuit system), which also includes a software configurable timer or phase-locked loop, for example, to generate a slave timing reference 120. The slave timing reference 120 is output to various components or peripherals of the slave device 104 for coordinated operation of the slave device 104.

[0023] Due to the effects of clock drift, master timing reference 106 and slave timing reference 120 may drift away from each other over time. In implementations requiring coordinated operation of master device 108 and slave device 104, the increasing desynchronization of timing between devices 104 and 108 is problematic.

[0024] The master timing reference 106 may be periodically broadcast into the wireless clock synchronization network 102 via a wireless radio transmitter (or transceiver) and associated antenna 114 via a data output 119. In various implementations, the master timing reference 106 may be a timestamp, or the slave control unit 118 may recover the master timing reference 106 based on the incoming packet rate. In addition, the wireless clock synchronization network 102 uses a dedicated data channel 121 (e.g., a designated frequency band within an existing device-to-device network or a completely independent network) for the master timing reference 106. In addition, the wireless clock synchronization network 102 may include other components such as access points, proxies, gateways, etc., as long as the total latency on the wireless clock synchronization network 102 is constant.

[0025] In some implementations, the wireless clock synchronization network 102 may be located in a frequency band shared with other device functionality within an existing device-to-device network, as long as bandwidth is guaranteed to be available within the wireless clock synchronization network 102 when the master timing reference 106 is issued. For example, burst transmissions (e.g., via a time division multiple access (TDMA) approach) with other functionality within the wireless network may be issued between periodic transmissions of the master timing reference 106. Furthermore, additional data may be combined with the periodic transmissions of the master timing reference 106 for additional functionality. As a result, the wireless clock synchronization network 102 may coexist on wireless networks used for other purposes.

[0026] One or more slave devices (e.g., slave device 104) are also connected to the wireless clock synchronization network 102 via a wireless radio receiver (or transceiver) and an associated antenna (e.g., receiver and antenna 117). As described above, the wireless clock synchronization network 102 may operate on a dedicated data channel 121 dedicated to clock synchronization, or share an existing wireless network with a guaranteed bandwidth for the master timing reference 106. In various implementations, the master timing reference 106 may reference the outgoing packet timestamp from the master device 108 (see, e.g., Figure 2 ), the incoming packet rate to the slave device 104 (see e.g. Figure 3 ), or other timing and / or delay reference points. Figure 1 102 is depicted as a wireless LAN, but in other implementations, the clock synchronization network 102 can be a wired LAN. In various implementations, the slave device 104 and the master device 108 are directly connected to the wireless clock synchronization network 102, as shown (also referred to as a direct wireless connection or a wireless connection).

[0027] When the slave device 104 receives the master timing reference 106 at the data input 123, the slave control unit 118 adjusts the slave timing reference 120 to match the master timing reference 106, including incorporating a predetermined correction factor based on at least a fixed known delay on a dedicated channel operating on the wireless clock synchronization network 102 between the devices 104, 108, without requiring RTT measurements that vary among the devices 104, 108. This results in synchronization of the timers / PLLs of each of the devices 104, 108 with extremely high timing synchronization accuracy.

[0028] In one example implementation, the crystal oscillator 110, 116 generates an electrical signal X at a frequency of 1 MHz. m , X s, and the associated timing circuit systems 112, 118 count the time ticks output from the crystal oscillators 110 and 116, which for a 1 MHz oscillator occur once every microsecond. As a result, the dedicated wireless clock synchronization network 102 can achieve extremely high timing synchronization accuracy, which in the context described herein can be within + / - 1 microsecond.

[0029] Figure 2 An example synchronization timeline 200 is shown that uses timestamps to synchronize slave devices 204 to a timing reference broadcast from a master device 208. The master device 208 includes a master crystal oscillator that generates an electrical signal X having a precisely known frequency. m .X m is input into the master control unit to generate the master timing reference 206. The master timing reference 206 is output to various components or peripherals of the master device 208 for coordinated operation of the master device 208.

[0030] Similarly, slave device 204 includes a slave crystal oscillator that generates an electrical signal X having a precisely known frequency. s . Electrical signal X s Similar to the electrical signal X m , but it is generated by the slave crystal oscillator independently of the master crystal oscillator. s is input into the slave control unit to generate the slave timing reference 220. The slave timing reference 220 is output to various components or peripherals of the slave device 204 for coordinated operation of the slave device 204.

[0031] For purposes of illustration, the master and slave crystal oscillators run at a frequency of 1 MHz. The 1 kHz timer outputs based on the master and slave crystal oscillators are considered the master timing reference 206 and the slave timing reference 220, respectively. Each of the 1 kHz timers of the timing references 206, 220 records a state change every 500 crystal ticks. The master and slave timing references 206, 220 are synchronized when their respective timers are reset to 0, but due to manufacturing and environmental differences, the master and slave timing references 206, 220 may diverge over time (e.g., although designed to operate at 1.00 Mhz, the slave crystal oscillator may actually operate at 1.01 MHz).

[0032] In this example, the master control unit includes a timing snapshot capture function (and associated modules) that captures a timing snapshot 230 of the master timing reference 206 at a predetermined value (here 200 ticks). The timing snapshot 230 includes at least the crystal tick value, but may also include additional information related to the master device 208 and / or the slave device 204. Although the timing snapshot 230 occurs at 200 crystal ticks, in other implementations, the timing snapshot 230 may occur at any number of crystal ticks and at any predetermined frequency (e.g., every 100 crystal ticks or every 2000 crystal ticks), depending on the application (e.g., the level of accuracy required and the rate at which the master and slave timing references 206, 220 diverge).

[0033] The timing snapshot 230 captured at 200 crystal ticks is transmitted to the slave device 204 via a dedicated clock synchronization channel for use in adjusting the slave timing reference 220 to match the master timing reference 206. This includes incorporating a predetermined correction factor or constant delay 232 (here 445 crystal ticks) based at least on a fixed known delay of a dedicated channel operating on a clock synchronization network between the devices 204, 208, without requiring RTT measurements that vary from device 204, 208 to device 204, 208. In various implementations, the constant delay 232 may be set and saved to the slave device 204 at the factory, or preset in the field at the slave device 204 for a particular pairing of the master device 208 and the slave device 204.

[0034] In this example, the constant delay 232 is calculated as a combination of the internal processing delay 234 at the master device 208 (e.g., the time required to process the timing snapshot 230 and send it out to the dedicated clock synchronization channel as an outgoing data packet 238), which is equal to 400 crystal ticks. In addition, an additional 45 crystal ticks are added to compensate for the transmission time between the master device 208 and the slave device 204 on the dedicated clock synchronization channel and the time required for the slave device 204 to receive and timestamp the incoming captured timing snapshot 230. The constant delay 232 is predetermined and pre-tested to remain constant for the time required for the captured timing snapshot 230 to be received by the slave device 204, and is known by the slave device 204 to perform timing adjustments. In some implementations, the constant delay 232 is only known to the master device 208 and is transmitted as part of the captured timing snapshot 230. In other implementations, the constant delay 232 is only known to the slave device 204 and does not have to be transmitted with the captured timing snapshot 230.

[0035] In this example, slave device 204 receives a captured timing snapshot 230 that is known to be captured at master device 208 at 200 crystal ticks. Slave device 204 also knows that its timestamp for receiving the captured timing snapshot should be equal to 645 crystal ticks (capture timer value: 200 + constant delay: 445). Slave device 204 compares the expected timestamp when receiving the captured timing snapshot (645 crystal ticks) with the actual timestamp 236 when receiving the captured timing snapshot (here 535 crystal ticks), resulting in a clock drift error of 110 crystal ticks. The slave control unit within slave device 204 then instructs slave timing reference 220 to switch its next state change 110 crystal ticks earlier than it would have otherwise (e.g., at adjusted state change 242, which is 890 crystal ticks = 1000-(200+445-535)), which results in an adjusted slave timing reference 240 that is synchronized to the captured timing snapshot of master device 208. In the illustrated case of the 1 kHz slave timing reference 220, the adjusted slave timing reference 240 resets to 0 crystal ticks at 890 rather than 1000 crystal ticks.

[0036] After the slave timing reference 220 is reset to the adjusted slave timing reference 240, the adjusted slave timing reference 240 will begin to drift again, resulting in a time gap that is resolved by periodically subsequently capturing timing snapshots at a sufficiently high packet rate. In other implementations, the subsequently captured timing snapshots are not periodic. For example, the subsequently captured timing snapshots may be pseudo-random, or defined by the available bandwidth on the master device 208 and / or the clock synchronization network. The timing of the subsequently captured timing snapshots may also be based on the detected error rate, the application state on the master device 208, or other considerations. They are repeatedly used to adjust the slave timing reference to maintain synchronization with the master device 208 (e.g., to prevent the time gap from exceeding an application-specific tolerance). In some implementations, the tolerance is used to determine the captured timing snapshot rate.

[0037] In various implementations, Figure 2The features of provide specific technical performance benefits to the disclosed timing synchronization system and method. Specifically, the timing snapshot 230 allows the slave timing reference 220 to be periodically reset to the adjusted slave timing reference 240, and maintains synchronization with the master device 208 within a tolerance that varies by application at a regular or irregular rate. The predetermined correction factor (also referred to herein as the constant delay 232) is predetermined and pre-tested to remain constant for the time required for the captured timing snapshot 230 to be received by the slave device 204, and is known by the slave device 204 to perform the timing adjustment. The predetermined correction factor is used in place of the RTT measurement, which has various disadvantages as described above. The predetermined correction factor includes a constant delay based on at least a fixed known delay of a dedicated channel operating on a clock synchronization network between the devices 204 and 208, including but not limited to delays on the processing and clock synchronization channels within the devices 204 and 208, without the need for RTT measurements that vary from device 204 to device 208.

[0038] Figure 3 Another example synchronization timeline 300 is shown for synchronizing a slave device 304 to a timing reference 306 (in the form of outgoing timing packets 342) that is periodically broadcast at a predetermined packet rate from a master device 308. The master device 308 includes a master crystal oscillator that generates an electrical signal X having a precisely known frequency. m .X m is input into the master control unit to generate the master timing reference 306. The master timing reference 306 is output to various components or peripherals of the master device 308 for coordinated operation of the master device 308.

[0039] Similarly, slave device 304 includes a slave crystal oscillator that generates an electrical signal X having a precisely known frequency. s . Electrical signal X s Similar to the electrical signal X m , but it is generated by the slave crystal oscillator independently of the master crystal oscillator. s is input into the slave control unit to generate the slave timing reference 320. The slave timing reference 320 is output to various components or peripherals of the slave device 304 for coordinated operation of the slave device 304.

[0040] For purposes of illustration, the master and slave crystal oscillators run at a frequency of 1 MHz. The 1 kHz timer outputs based on the master and slave crystal oscillators are considered the master timing reference 306 and the slave timing reference 320, respectively. Each of the 1 kHz timers of the timing references 306, 320 records a state change every 500 crystal ticks. The master and slave timing references 306, 320 are synchronized when their respective timers are reset to 0, but due to manufacturing and environmental differences, the master and slave timing references 306, 320 may diverge over time (e.g., although designed to operate at 1.00 Mhz, the slave crystal oscillator may actually operate at 1.01 MHz).

[0041] The master control unit includes an outgoing packet transmission feature that sends outgoing timing packets 342 over a dedicated clock synchronization channel for adjusting the slave timing reference 320 to match the master timing reference 306. The outgoing timing packets 342 are sent at a known (at least known to the slave device 304) packet rate (here, every 1000 crystal ticks), however, the packet rate can be any predetermined frequency (e.g., every 500 crystal ticks or every 2000 crystal ticks), depending on the application (e.g., depending on the level of accuracy required and the rate at which the master timing reference and the slave timing reference diverge). In various implementations, some outgoing timing packets 342 are lost during transmission. Some data packet losses can be tolerated by not correcting for data packet losses when they occur.

[0042] A slave control unit within slave device 304 includes a timing feedback control module (e.g., a proportional-integral-derivative (PID) controller) that compares the measured crystal ticks between received timing packets 342 to the known packet rate of timing reference 306. The slave control unit then adjusts slave timing reference 320 over time to match the timing reference 342 of the outgoing timing packets. For example, Figure 3 As shown, the outgoing timing reference 306 packet rate is set to 1000 crystal ticks. However, the first outgoing timing packet 342 is received at the slave device 304 at 890 crystal ticks. If 890 of the 1000 crystal ticks exceed the acceptable synchronization tolerance between the devices 304, 308, this will trigger the slave control unit to run a synchronization algorithm to adjust the slave timing reference 320 over time.

[0043] Specifically, the timing feedback control module adjusts its next 1000 crystal ticks to restart at 785 instead of 1000 to compensate for the accumulated clock drift and the expected future clock drift. The subsequent timing packet 342 instructs the timing feedback control module to adjust its next three 1000 crystal tick cycles to restart at 875, 900, and 890, thereby sequentially iteratively synchronizing the slave timing reference 320 to the master timing reference 306. At the synchronization point 344, the slave timing reference 320 is synchronized to the master timing reference 306, and subsequent crystal tick cycles restart at 890 until the synchronization between the devices 304, 308 is again outside the acceptable synchronization tolerance. This again triggers the slave control unit to run the synchronization algorithm to again adjust the slave timing reference 320 over time.

[0044] To the extent that the slave timing reference 320 remains synchronized with the master timing reference 306, the subsequent 1000 crystal tick cycles restart at 890 of the slave timing reference 320 to adjust only for expected future clock drift. The timing packets 342 are repeatedly used to adjust the slave timing reference 320 to maintain synchronization with the master timing reference 306 to prevent the time gap from exceeding an application-specific tolerance. The tolerance is predetermined to produce the captured timing snapshot rate.

[0045] Although synchronized in frequency, master timing reference 306 and slave timing reference 320 may not be edge aligned, indicating that they are out of phase after the above-mentioned packet rate adjustment at slave device 304. To achieve approximate edge alignment, slave device 304 may utilize packet reception times to edge align with master timing reference 30, optionally taking into account known fixed transmission delays.

[0046] Figure 2 and 3 Not drawn to scale. Clock drift may be exaggerated compared to what might actually occur to illustrate the technical solution. In various implementations, Figure 2 Output packet 238, and subsequent output packets, and Figure 3 The timing packets 342 are sent out via the dedicated data channel. The dedicated data channel allows the latency of receiving packets to be relatively constant, which allows the present application to perform timing adjustments without RTT measurements.

[0047] Figure 4A dedicated wireless clock synchronization network 402 of a slave speaker 404 synchronized to a master speaker 408 is shown. In an example implementation, the speakers 404, 408 form a fully synchronized multi-channel surround wireless speaker system. The master speaker 408 generates a master timing reference, while the slave speaker 404 similarly generates a slave timing reference. Due to the effects of clock drift, the master timing reference 106 and the slave timing reference 120 will drift away from each other over time. In implementations that require coordinated operation of the master speaker 408 and the slave speaker 404, such as synchronized audio output, the timing between the speakers 404 and 408 becoming increasingly out of sync is problematic.

[0048] The master timing reference is periodically broadcast into the wireless clock synchronization network 402 via a wireless radio transmitter and associated antenna. The slave speaker 404 is also connected to the wireless clock synchronization network 402 via a wireless radio receiver and associated antenna (or transceiver). The wireless clock synchronization network 402 may operate on a dedicated channel dedicated to clock synchronization. In other implementations, the wireless clock synchronization network 402 may share an existing wireless network with a guaranteed bandwidth for the master timing reference. In various implementations, the master timing reference may reference the outgoing packet timestamp from the master speaker 408 (see, e.g., Figure 2 ), to the incoming packet rate from speaker 404 (see e.g. Figure 3 ) or other timing reference point. Figure 4 Although depicted as a wireless LAN, in other implementations, the clock synchronization network 402 may be a wired LAN.

[0049] When a master timing reference is received from speaker 404, the slave timing reference is adjusted to match the master timing reference, including incorporating a predetermined correction factor based on a fixed known delay of a dedicated channel operating on the wireless clock synchronization network 402 between at least speakers 404, 408, without requiring RTT measurements that vary between speakers 404, 408.

[0050] Figure 5 A dedicated wireless clock synchronization network 502 of slave microphones 504 synchronized to a master microphone 508 is shown. In an example implementation, the microphones 504, 508 form a fully synchronized microphone array to capture audio from a user 522. In some implementations, the microphones 504, 508 utilize Sound Source Localization (SSL) to estimate the location of the user 522 using the relative time delays of the sounds arriving at each microphone 504, 508. The more accurate the clock synchronization of the microphones 504, 508, the more accurate the SSL can be.

[0051] The master microphone 508 generates a master timing reference, while the slave microphone 504 similarly generates a slave timing reference. Due to the effects of clock drift, the master timing reference 106 and the slave timing reference 120 will drift away from each other over time. In implementations that require coordinated operation of the master microphone 508 and the slave microphone 504, such as synchronized audio input, the increasing desynchronization of the timing between the microphones 504 and 508 is problematic.

[0052] The master timing reference is periodically broadcast into the wireless clock synchronization network 502 via a wireless radio transmitter and associated antenna. The slave microphone 504 is also connected to the wireless clock synchronization network 502 via a wireless radio receiver and associated antenna (or transceiver). The wireless clock synchronization network 502 may operate on a dedicated channel dedicated to clock synchronization. In other implementations, the wireless clock synchronization network 502 may share an existing wireless network with a guaranteed bandwidth for the master timing reference. In various implementations, the master timing reference may reference the outgoing packet timestamp from the master microphone 508 (see, e.g., Figure 2 ), to the incoming packet rate from microphone 504 (see e.g. Figure 3 ) or other timing reference point. Figure 5 Although depicted as a wireless LAN, in other implementations, the clock synchronization network 502 may be a wired LAN.

[0053] When a master timing reference is received from microphone 504, the slave timing reference is adjusted to match the master timing reference, including incorporating a predetermined correction factor based on a fixed known delay of a dedicated channel operating on the wireless clock synchronization network 502 between at least microphones 504, 508, without requiring RTT measurements that vary between microphones 504, 508.

[0054] Figure 6 A dedicated wireless clock synchronization network 602 of a slave camera 604 synchronized to a master camera 608 is shown. In an example implementation, the cameras 604, 608 form a fully synchronized stream from the independently recording cameras 604 and 608. In some implementations, the cameras 604, 608 are used to output seamless camera feed transitions, seamless image stitching, and / or depth sensing, each of which operates more seamlessly with more accurate clock synchronization of the cameras 604, 608.

[0055] The master camera 608 generates a master timing reference, while the slave camera 604 similarly generates a slave timing reference. Due to the effects of clock drift, the master timing reference 106 and the slave timing reference 120 will drift away from each other over time. In implementations that require coordinated operation of the master camera 608 and the slave camera 604, such as synchronized video output, the increasingly unsynchronized timing between the cameras 604 and 608 is problematic.

[0056] The master timing reference is periodically broadcast into the wireless clock synchronization network 602 via a wireless radio transmitter and associated antenna. The slave cameras 604 are also connected to the wireless clock synchronization network 602 via wireless radio receivers and associated antennas (or transceivers). The wireless clock synchronization network 602 may operate on a dedicated channel dedicated to clock synchronization. In other implementations, the wireless clock synchronization network 602 may share an existing wireless network with a guaranteed bandwidth for the master timing reference. In various implementations, the master timing reference may reference the outgoing packet timestamp from the master camera 608 (see, e.g., Figure 2 ), to the incoming packet rate from camera 604 (see e.g. Figure 3 ) or other timing reference point. Figure 6 Although depicted as a wireless LAN, in other implementations, the clock synchronization network 602 may be a wired LAN.

[0057] When the master timing reference is received from camera 604, the slave timing reference is adjusted to match the master timing reference, including incorporating a predetermined correction factor based on a fixed known delay of a dedicated channel operating on the wireless clock synchronization network 602 between at least cameras 604, 608, without requiring RTT measurements that vary between cameras 604, 608.

[0058] The technology disclosed herein is applicable to various audio and / or video input and / or output devices, such as Figure 3-6 The technology of the present disclosure may also be applicable to various devices outside the context of audio and / or video. For example, distributed computing applications typically require a high level of synchronization between computing devices. In addition, in a test or manufacturing environment with multiple sensors and complex feedback control mechanisms, high sampling rates and synchronization in time and space may be required to coordinate sensor results in rapidly changing systems (e.g., in self-driving cars, industrial automation and control systems, aviation systems, smart homes, monitoring systems, etc.). More generally, any application that requires particularly high sampling rates and / or synchronization in time and space can benefit from the technology of the present disclosure.

[0059] The technology of the present disclosure can utilize dedicated communication channels and dedicated processing capabilities to achieve substantially fixed waiting times without RTT measurements. In various implementations, the dedicated communication channels and dedicated processing capabilities are fully implemented in hardware and / or firmware. In addition, software can be used to reserve processing power and / or bandwidth within shared hardware resources to achieve similar substantially fixed waiting times without dedicated separate hardware resources and without RTT measurements.

[0060] The technology of the present disclosure can use programmable logic blocks, which can interconnect independent peripheral devices with fixed waiting time. Example representative programmable logic devices are field programmable gate arrays (FPGAs) and complex programmable logic devices (CPLDs). The synthesis of hardware logic blocks can force timing behavior, which ensures fixed waiting time by design. In addition, some microcontroller architectures support autonomous peripheral operations that simulate FPGA and CPLD operations, which can be used to implement the technology of the present disclosure. Hard wiring the events and tasks of independent internal peripherals can minimize waiting time. Once configured at startup, the events generated by the peripherals can autonomously trigger another peripheral device at a fixed waiting time without the intervention of any processing element.

[0061] In various implementations, the dedicated wireless side channel described herein can be implemented using existing radios in the master and slave devices as long as the radios support constant time implementation and have the ability to share existing radio resources for clock synchronization. For example, Wi-Fi chipsets and communication stacks can support ad hoc network construction, while Bluetooth chipsets and communication stacks can provide users with full control over radios and other constrained peripherals.

[0062] The dedicated wireless side channel described herein can be extended to multi-hop dedicated networks at the expense of cumulative errors per hop (e.g., a 5-hop network may have approximately five time ticks of accuracy). Multi-hop networks may employ a broadcast-based managed flooding technique, where each retransmission has a constant retransmission processing delay subtracted from the timestamp before being relayed. For example, an indicator may be embedded in the master timing reference to indicate the number of hops made and the maximum number of hops possible (e.g., by counting backwards from the maximum value).

[0063] Figure 7 An example operation 700 is shown for synchronizing a slave device to a master device. A generation operation 705 generates a master timing reference using master timing circuitry. A transmission operation 710 transmits the master timing reference from the master device to a dedicated data channel. In some embodiments, the dedicated data channel is a wireless data channel and is transmitted using a wireless radio transmitter.

[0064] A receiving operation 715 receives a master timing reference from a master device via a dedicated data channel. In some implementations, the dedicated data channel is a wireless data channel and is received using a wireless radio receiver. The received master timing reference has a fixed known delay caused by delays within one or more of the master device, the dedicated data channel, and the slave device.

[0065] The synchronization operation 720 synchronizes the slave timing reference to the master timing reference using a correction factor based on a fixed known delay. In some implementations, the synchronization operation 720 uses a snapshot of the master timing reference and a predetermined delay of a data packet to be transmitted between the master device and the slave device containing the timing snapshot. In other implementations, the synchronization operation 720 uses a known packet rate of a series of incoming packets containing the master timing reference from the master device. Operation 700 can be repeated iteratively and automatically to continuously synchronize the slave device to the master device.

[0066] Figure 8 Example hardware and software that may be useful in implementing the described techniques are shown. The master and / or slave devices disclosed herein may be remote control devices and / or physically controlled devices, and may be network connected and / or network-enabled devices, and may be client devices such as laptops, mobile devices, desktops, tablets, servers / cloud devices, IoT devices; for example, electronic accessories or other electronic devices.

[0067] The master device and / or slave device disclosed herein may each include a system board 800 on which various microelectronic components of the device are attached and interconnected. For example, the system board 800 may include one or more processors 802 (e.g., discrete or integrated microelectronic chips and / or separate but integrated processor cores, including but not limited to: a central processing unit (CPU) and a graphics processing unit (GPU) and at least one memory device 804 (which may be integrated into the system or chip of the device). The master device and / or the slave device may also include a storage medium 820 (e.g., a flash memory or hard drive), one or more displays 822, and other input / output (I / O) devices 834. The input / output (I / O) devices 834 may allow a user to input commands and information (e.g., via a keyboard or mouse). These and other input devices may be coupled to a server via one or more I / O interfaces 838 such as a serial port interface, a parallel port, and / or a universal serial bus (USB).

[0068] The memory device 804 and the storage medium 820 may include one or both of volatile memory (e.g., random access memory (RAM)) and non-volatile memory (e.g., flash memory or magnetic storage). The operating system 810 (such as various Microsoft The operating system 810 resides in the memory device 804 and / or the storage medium 820 and is executed by at least one of the processor units 802, but other operating systems may also be used. One or more additional applications 850 are loaded into the memory device 804 and / or the storage medium 820 and are executed by at least one of the processor units 802 within the operating system 810.

[0069] The system board 800 may also include (or be connected to an external) power supply 816, which is powered by one or more batteries or other power sources and provides power to the system board 800 and associated master and / or slave devices. The power supply 816 may also be connected to an external power source for overriding or recharging the battery.

[0070] The system board 800 may also include one or more communication transceivers 830, which may be connected to one or more antennas 832 to provide network connectivity (e.g., mobile phone networks, ). The communication device 800 may also include a network adapter 836, which is a type of communication interface. The system board 800 may use the network adapter 836 and any other type of communication device to establish a connection on a wide area network (WAN) or a local area network (LAN). The master device and / or slave device described herein may each include a communication transceiver 830 connected to an antenna 832 to send and / or receive master timing signals through a dedicated wireless side channel. The network connections shown are exemplary, and other communication devices and means may be used to establish a communication link between the master device and / or slave device.

[0071] The master device and / or the slave device may include various tangible computer-readable storage media (e.g., memory devices and storage media devices) and intangible computer-readable communication signals. Tangible computer-readable storage can be implemented by any available medium accessible to the master device and / or the slave device, and includes both volatile and non-volatile storage media, and removable and non-removable storage media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Tangible computer-readable storage media include, but are not limited to, RAM, read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical disk storage, cassettes, tapes, disk storage or other magnetic storage devices, or any other tangible media that can be used to store desired information and can be accessed by the master device and / or the slave device. Tangible computer-readable storage media do not include intangible communication signals.

[0072] Intangible computer-readable communication signals may implement computer-readable instructions, data structures, program modules, or other data residing in a modulated data signal, such as a carrier wave or other signal transmission mechanism. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in a manner that encodes information in the signal. By way of example and not limitation, intangible communication signals include signals that travel through wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, radio frequency (RF), infrared (IR) and other wireless media.

[0073] Some embodiments may include articles of manufacture. Articles of manufacture may include tangible storage media for storing logic. Examples of storage media may include one or more types of computer-readable storage media capable of storing electronic data, including volatile or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, and the like. Examples of logic may include various software elements, such as software components, programs, applications, computer programs, applications, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, operating segments, methods, procedures, software interfaces, application program interfaces (APIs), instruction sets, computing codes, computer codes, code segments, computer code segments, words, values, symbols, or any combination thereof. For example, in one embodiment, articles of manufacture may store executable computer program instructions that, when executed by a computer, cause the computer to perform methods and / or operations according to the described embodiments. Executable computer program instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, and the like. Executable computer program instructions can be implemented according to a predefined computer language, manner or syntax for instructing a computer to perform a specific operation segment. These instructions can be implemented using any suitable high-level, low-level, object-oriented, visual, compiled, and / or interpreted programming language.

[0074] Some embodiments of the present invention described herein can be implemented as logical steps in one or more computer systems. Logical operations are implemented as (1) a sequence of steps implemented by a processor executed in one or more computer systems and (2) interconnected machines or circuit modules within one or more computer systems. This implementation is a matter of choice depending on the performance requirements of the computer system implementing the present invention. Therefore, the logical operations described herein are variously referred to as operations, steps, objects, or modules. In addition, the logical operations can be performed in any order, added as needed, or ignored, unless otherwise explicitly stated or the claim language inherently requires a specific order.

[0075] An example clock synchronization network according to the techniques of the present disclosure includes a master device and a slave device. The master device includes a master timing circuit system for generating a master timing reference and a data output for transmitting the master timing reference on a dedicated data channel. The slave device includes a data input for receiving the master timing reference on the dedicated data channel and a slave timing circuit system for synchronizing the slave timing reference to the master timing reference using a predetermined delay on the dedicated data channel.

[0076] In another example clock synchronization network according to the techniques of this disclosure, the slave timing circuit system synchronizes the slave timing reference to the master timing reference using a snapshot of the master timing reference and a predetermined delay of a data packet containing the snapshot to be transmitted between the master device and the slave device.

[0077] In another example clock synchronization network in accordance with the techniques of this disclosure, the slave timing circuitry synchronizes the slave timing reference to the master timing reference using a known packet rate of a series of incoming data packets from the master device containing the master timing reference.

[0078] In yet another example clock synchronization network in accordance with the techniques of this disclosure, the master timing circuitry includes an oscillator used as a reference for a timer that generates the master timing reference.

[0079] In yet another example clock synchronization network according to the techniques of the present disclosure, the slave timing circuitry includes an oscillator used as a reference for a timer that generates the slave timing reference, and wherein the slave timing circuitry synchronizes the slave timing reference by adjusting an oscillator count to switch the slave timing reference.

[0080] In yet another example clock synchronization network according to the techniques of this disclosure, the predetermined delay includes a delay at one or more of: the master device, the slave device, and the dedicated data channel.

[0081] In yet another example clock synchronization network according to the techniques of this disclosure, the slave timing reference is synchronized to the master timing reference without requiring round trip time (RTT) measurements.

[0082] In yet another example clock synchronization network according to the techniques of this disclosure, the dedicated data channel is a direct wireless connection between the master device and the slave devices.

[0083] In yet another example clock synchronization network according to the techniques of this disclosure, the dedicated data channel is a wired connection between the master device and the slave devices.

[0084] Another example clock synchronization network according to the techniques of this disclosure also includes a second slave device. The second slave device includes a data input for receiving a master timing reference from the slave device on a dedicated data channel, and slave timing circuitry for synchronizing the second slave timing reference to the master timing reference using a correction factor based on a predetermined delay on the dedicated data channel and a number of hops from the master device to the second slave device.

[0085] In yet another example clock synchronization network according to the techniques of this disclosure, the master device is a component of an autonomous vehicle and the slave device is a sensor on the autonomous vehicle.

[0086] In yet another example clock synchronization network according to the techniques of this disclosure, the master device and the slave devices are each one or both of audio and video devices.

[0087] In yet another example clock synchronization network according to the technology of this disclosure, the synchronization accuracy of the slave timing reference and the master timing reference is within one time tick.

[0088] In yet another example clock synchronization network according to techniques of this disclosure, a master timing reference is generated by an application executing on a master device.

[0089] In yet another example clock synchronization network in accordance with techniques of this disclosure, a master timing reference is generated at a rate based on a required accuracy of an application executing on a slave device.

[0090] A method for synchronizing a slave device to a master device according to the technology of the present disclosure includes: using a master timing circuit system to generate a master timing reference; transmitting the master timing reference from the master device to the outside through a dedicated data channel; receiving the master timing reference with a predetermined delay from the master device to the slave device through the dedicated data channel; and synchronizing the slave timing reference of the slave device to the master timing reference using the predetermined delay on the dedicated data channel.

[0091] In another example method for synchronizing a slave device to a master device according to techniques of this disclosure, the synchronization operation uses a snapshot of a master timing reference and a predetermined delay of data packets to be transmitted between the master and slave devices containing the snapshot.

[0092] In yet another example method for synchronizing a slave device to a master device in accordance with the techniques of this disclosure, the synchronization operation uses a known packet rate of a series of incoming packets from the master device that contain a master timing reference.

[0093] In yet another example method for synchronizing a slave device to a master device in accordance with the techniques of this disclosure, the dedicated data channel is a wireless connection between the master device and the slave device.

[0094] Yet another example clock synchronization network according to the techniques of the present disclosure includes a master device and a slave device. The master device includes a master timing circuit system for generating a master timing reference and a wireless radio transmitter for transmitting the master timing reference with a fixed known delay on a dedicated wireless data channel. The slave device includes a wireless radio receiver for receiving the master timing reference on the dedicated wireless data channel and a slave timing circuit system for synchronizing the slave timing reference to the master timing reference with a correction factor based on the fixed known delay.

[0095] The above description, examples and data provide a complete description of the structure and use of exemplary embodiments of the present invention. Because many embodiments of the present invention can be made without departing from the spirit and scope of the present invention, the present invention falls within the scope of the appended claims. In addition, the structural features of different embodiments can be combined with another embodiment without departing from the claims recorded.

Claims

1. A clock synchronization network, comprising: Main equipment, including: master timing circuitry for generating a master timing reference; and a data output for transmitting the master timing reference on a dedicated data channel; and From the device, including: for receiving a data input of the master timing reference on the dedicated data channel; and A slave timing circuit system for synchronizing a slave timing reference to the master timing reference using a constant delay on the dedicated data channel, wherein the slave timing circuit system synchronizes the slave timing reference to the master timing reference using a snapshot of the master timing reference transmitted on the dedicated data channel and a fixed known delay of a data packet containing the snapshot to be transmitted between the master device and the slave device.

2. The clock synchronization network according to claim 1, characterized in that: The master timing circuitry includes an oscillator that is used as a reference for a timer that generates the master timing reference.

3. The clock synchronization network according to claim 1, characterized in that: The slave timing circuitry includes an oscillator that serves as a reference for a timer that generates the slave timing reference, and wherein the slave timing circuitry synchronizes the slave timing reference by adjusting an oscillator count to switch the slave timing reference.

4. The clock synchronization network according to claim 1, characterized in that: The predetermined delay includes a delay at one or more of: the master device, the slave device, and the dedicated data channel.

5. The clock synchronization network according to claim 1, characterized in that: The slave timing reference is synchronized to the master timing reference without requiring round trip time (RTT) measurements.

6. The clock synchronization network according to claim 1, characterized in that: The dedicated data channel is a direct wireless connection between the master device and the slave device.

7. The clock synchronization network according to claim 1, characterized in that: The dedicated data channel is a wired connection between the master device and the slave device.

8. The clock synchronization network according to claim 1, characterized in that: Also includes: The second slave device comprises: for receiving a data input of said master timing reference from said slave device on said dedicated data channel; as well as Slave timing circuitry for synchronizing a second slave timing reference to the master timing reference using a correction factor based on a constant delay on the dedicated data channel and a number of hops from the master device to the second slave device.

9. The clock synchronization network according to claim 1, characterized in that: The master device is a component of an autonomous vehicle, and the slave device is a sensor on the autonomous vehicle.

10. The clock synchronization network according to claim 1, characterized in that: The master device and the slave device are each one or both of an audio and a video device.

11. The clock synchronization network according to claim 1, characterized in that: The slave timing reference is synchronized with the master timing reference with an accuracy within one time tick.

12. The clock synchronization network according to claim 1, characterized in that: The master timing reference is generated by an application executing on the master device.

13. The clock synchronization network according to claim 12, characterized in that: The master timing reference is generated at a rate based on the required accuracy of the application executing on the slave device.

14. A method for synchronizing a slave device to a master device, comprising: using a master timing circuit system to generate a master timing reference; transmitting the master timing reference outwardly from the master device via a dedicated data channel; receiving the master timing reference with a constant delay from the master device to the slave device on the dedicated data channel; as well as A slave timing reference of the slave device is synchronized to the master timing reference using a constant delay on the dedicated data channel, the synchronization operation using a snapshot of the master timing reference transmitted on the dedicated data channel and a fixed known delay of a data packet to be transmitted between the master device and the slave device containing the snapshot.

15. The method according to claim 14, characterized in that The dedicated data channel is a wireless connection between the master device and the slave device.

Images