Multi-chiplet clock delay compensation

Abstract

A method and system for clock delay compensation in a multi-chiplet system is disclosed. The disclosed technique includes a clock generator distributing a clock signal across a distribution tree of each chiplet, and a phase detector measuring phase, where each phase measurement is associated with one of the chiplets and indicates a propagation speed of the clock signal through the distribution tree of the chiplet. Then, for each chiplet, a microcontroller determines a delay offset based on the phase measurement associated with the chiplet, and delays the propagation of the clock signal through the distribution tree of the chiplet using a delay unit associated with the chiplet based on the delay offset.

Description

[Technical field]

[0001] (Related Applications) This application claims the benefit of U.S. Patent Application No. 17 / 565,382, filed December 29, 2021, the contents of which are incorporated herein by reference. [Background technology]

[0002] The propagation of data across cascading logic units is usually controlled by a clock signal. When interface logic units are integrated into two chips, data must be exchanged across chips via wires. Due to physical limitations in the density of connecting wires (e.g., 100 wires per millimeter), a physical layer interface is used to drive data between logic units. The physical layer is usually designed to operate at a higher data rate than the native data rate of the logic units it connects to allow more data to be transferred over a limited number of connections. However, using a physical layer brings overhead due to their complex circuitry and increased power consumption.

[0003] New packaging techniques allow for an increased number of connections between logic units within and between chips (e.g., 10,000 wires per millimeter), so that more logic units can be packed onto a chip and more chips can be assembled within a chip package. In turn, the increased density in logic circuits and their connections requires simpler mechanisms for connecting logic units. A simple solution is to make direct connections between the outputs and inputs of logic units. However, such direct connections, while saving the overhead associated with the use of a physical layer, require synchronized clock distribution; that is, the clock signal propagating across cascaded logic units needs to be synchronized to enable proper operation.

[0004] A more detailed understanding may be had from the following description, given by way of example in conjunction with the accompanying drawings, in which: [Brief description of the drawings]

[0005] [Figure 1] 1 is a block diagram of an example device in which one or more features of the present disclosure may be implemented. [Diagram 2] 1 is a schematic diagram of an example clock distribution to logic units in which one or more features of the present disclosure may be implemented. [Diagram 3] 1 is a schematic diagram of an example clock distribution tree across a chip in which one or more features of the present disclosure may be implemented. [Figure 4] 1 is a schematic diagram of an example multi-chiplet in which one or more features of the present disclosure may be implemented. [Diagram 5] 1 is a block diagram of an example system for clock delay compensation in which one or more features of the present disclosure may be implemented. [Figure 6] 1 is a flowchart of an example method for clock delay compensation in which one or more features of the present disclosure may be implemented. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] Systems and methods are provided for clock delay compensation in multi-chip systems. Clock signals propagating through the clock distribution trees of the chiplets of the system are used to control the synchronized operation of cascaded logic units. Variations in hardware manufacturing and environmental conditions cause variations in the propagation speed of the clock signals in different chiplets and in different parts of the distribution tree within each chiplet. As a result, the clock signals are not aligned (i.e., they are delayed relative to each other) when they arrive at the clock inputs of the circuits that depend on them. The techniques described herein monitor phase measurements associated with these misalignments and, based on these phase measurements, periodically adjust the delays applied to the clock signals to compensate for the misalignments.

[0007] Aspects disclosed in this disclosure describe a method for clock delay compensation in a multi-chiplet system. The method includes distributing a clock signal across a distribution tree of each chiplet by a clock generator and measuring a phase by a phase detector, where each phase measurement is associated with a chiplet and indicates a propagation speed of the clock signal through the distribution tree of the chiplet. Then, for each chiplet, the method includes determining, by a microcontroller, a delay offset based on the phase measurement associated with the chiplet, and delaying propagation of the clock signal through the distribution tree of the chiplet using a delay unit associated with the chiplet based on the delay offset.

[0008] Aspects disclosed in the present disclosure also describe a multi-chiplet system for clock delay compensation. The system includes at least one processor and a memory that stores instructions. The instructions, when executed by the at least one processor, cause the system to distribute, by a clock generator, a clock signal across a distribution tree of each chiplet, and measure a phase by a phase detector, where each phase measurement is associated with a chiplet and indicates a propagation speed of the clock signal through the distribution tree of the chiplet. Then, for each chiplet, the instructions further cause the system to determine, by a microcontroller, a delay offset based on the phase measurement associated with the chiplet, and delay propagation of the clock signal through the distribution tree of the chiplet using a delay unit associated with the chiplet based on the delay offset.

[0009] Further aspects disclosed herein describe a non-transitory computer-readable storage medium including instructions executable by at least one processor for implementing a method for clock delay compensation in a multi-chiplet system. The method includes distributing, by a clock generator, a clock signal across a distribution tree of each chiplet, and measuring a phase by a phase detector, where each phase measurement is associated with a chiplet and indicates a propagation speed of the clock signal through the distribution tree of the chiplet. Then, for each chiplet, the method includes determining, by a microcontroller, a delay offset based on the phase measurement associated with the chiplet, and delaying propagation of the clock signal through the distribution tree of the chiplet using a delay unit associated with the chiplet based on the delay offset.

[0010] 1 is a block diagram of an example device 100 in which one or more features of the present disclosure may be implemented. Device 100 may be, for example, a computer, a gaming device, a handheld device, a set-top box, a television, a mobile phone, or a tablet computer. Device 100 may include a processor 110, an accelerated processing unit (APU) 120, storage 130, input devices 140, memory 150, and output devices 160. Device 100 may also include input drivers 145 and output drivers 165. Processor 110 and APU 120 may each represent one or more central processing unit (CPU) cores and one or more APU cores. Memory 150 may represent volatile or non-volatile memory including random-access memory (RAM), SRAM, dynamic random access memory (DRAM), cache, or combinations thereof. The processor 110, the APU 120, the memory 150, or a subset thereof, may be located on the same die or on separate dies. In an aspect, the device 100 may include additional components not shown in FIG.

[0011] APU 120 may represent a graphics processing unit (GPU), i.e., a shader system with one or more parallel processing units configured to perform computations according to, for example, the single instruction multiple data (SIMD) paradigm. APU 120 may be configured to accept computational and image rendering commands from processor 110, process the computational and image rendering commands, and / or provide output to a display (output device 160).

[0012] Storage 130 may include fixed or removable storage (e.g., hard disk drive, solid state drive, optical disk, flash drive). Input device 140 may represent, for example, a keyboard, a keypad, a touch screen, a touch pad, a detector, a microphone, an accelerometer, a gyroscope, a biometric scanner, or a network connection (e.g., a wireless local area network card for receiving wireless IEEE 802 signals). Output device 160 may include, for example, a display, a speaker, a printer, a haptic feedback device, one or more lights, antennas, or a network connection (e.g., a wireless local area network card for transmitting wireless IEEE 802 signals). In one aspect, input driver 145 communicates with processor 110 (or APU 120) and input device 140 to facilitate receipt of input from input device 140 to processor 110 (or APU 120). In another aspect, the output driver 165 communicates with the processor 110 (or APU 120 ) and the output device 160 and facilitates transmission of output from the processor 110 (or APU 120 ) to the output device 160 .

[0013] FIG. 2 is a schematic diagram of an example clock distribution to a logic unit 200 in which one or more features of the present disclosure may be implemented. Current integrated circuit technology allows for the integration of cascaded logic units, so that each logic unit performs a computation operation on data provided to it from the previous logic unit and then supplies the processed data to the next logic unit. For example, three logic units 220.1-220.3 may be designed to perform a pipelined computation, as shown in FIG. 2. An input (or send) register 205.1 may hold input data that is operated on by the circuit 210.1 of the first logic unit, and the result is stored in an output (or capture) register 215.1. The captured data may be transferred from that output register 215.1 to an input register 205.2 of a second logic unit 220.2 in the pipeline. Upon completion of its computation, the second logic unit's circuit 210.2 outputs the resulting data to its output register 215.2, which is transferred to the input register 205.3 of the third logic unit 220.3. Similarly, upon completion of its computation, the third logic unit's circuit 210.3 outputs the resulting data to its output register 215.3, which is transferred down the pipeline to the input register of the subsequent logic unit.

[0014] To perform pipelined processing, as mentioned above, each logic unit needs to be informed when it can start its operation, i.e., when input data is ready in its input register for the logic circuit to operate on. For that purpose, a clock signal generated by clock generator 230 may be distributed to registers 205.1-205.3, 215.1-215.3. To ensure proper operation of the cascaded logic units, the distributed clock signal must be aligned as shown by clock signals 240.1 and 240.2 when measured at the clock input (herein referred to as a sink point) of the circuit that requires it. For example, logic unit 220.1 may be configured to start its operation based on data stored in input register 205.1 at the rising edge of clock pulse 242 measured at sink point 230.1, and store the resulting data in its output register 215.1. The data stored in output register 215.1 can then be released (transferred) to input register 205.2 of logic unit 220.2 on the rising edge of clock pulse 244 measured at sink point 230.2. As long as the arrival times of the clock signals at sink point 230.1 and sink point 230.2 are sufficiently similar, logic 210.1 will be able to complete its computation and store the resulting data in its output register 215.1 when it receives the input data, before the rising edge of clock pulse 244 at sink point 230.2 (pulse rising edge 244 that triggers the transfer of data between output register 215.1 and input register 205.2). Thus, if the clock signals (measured at the sink points) are synchronized, the processed data will propagate properly through the pipeline if each logic unit will complete its operation within the computation time 246 that is in a clock cycle.In another example, if two logic units, e.g. 220.2 and 220.3, are integrated on different chips, the transfer of data from output register 215.2 to input register 205.3 may be triggered by the rising edge of a clock pulse at sink point 230.3 and by the rising edge of a clock pulse at sink point 230.4, respectively. Even in such a case, the clock signals measured at sink points 230.3 and 230.4 need to be synchronized to allow the processed data to propagate properly across the chip.

[0015] However, in reality, the clock signals generated by clock generator 230 and distributed along the clock network (or clock distribution tree) do not arrive at the circuits (sink points) that consume them at the same time. For example, a clock signal distributed along a path starting from clock generator 230 and ending at sink point 230.1 may be delayed in time with respect to a clock signal distributed along a path starting from clock generator 230 and ending at sink point 230.2, as shown by clock signals 250.1 and 250.2. This time delay, i.e., clock skew 258, includes a static component and a dynamic component. The static component may be caused by the difference in length between the two paths that follow from clock generator 230 to the location of their respective sink points. The dynamic component may be caused by manufacturing variations that cause the hardware through which the clock signals pass to change their electrical properties (e.g., conductivity) differently than temperature and voltage variations. Therefore, when designing the cascaded logic units, e.g., 220.1-220.3, clock skew 258 must be taken into account to accommodate both the computation time 256 and the expected maximum clock skew 258 in a clock cycle. In other words, logic units 220.1-220.3 must be designed to complete their operations and load their output registers 215.1-215.3 before the next rising edge of clock pulse 254. Such rising edge of clock pulse 254 may occur earlier compared to rising edge of clock pulse 244 when the clock signals are synchronized. Therefore, the clock signals must be sufficiently aligned (synchronized) so that when they reach the sink points (e.g., 230.1-230.5) in the chip, the resulting clock skew is within a tolerance below the expected maximum clock skew 258.

[0016] FIG. 3 is a schematic diagram of an exemplary clock distribution tree across a chip 300 in which one or more features of the present disclosure may be implemented. FIG. 3 illustrates a chip (e.g., a 20 mm2 chip 305) that includes logic units 340 that interface via registers (e.g., logic units 220.1-220.3 shown in FIG. 2). The chip also includes a clock distribution tree 320, the root of which is fed with a clock signal generated by a clock generator 310. The generated clock signal is split into clock signals that propagate across the branches of the tree to reach circuits designed to be triggered by the arrival of the clock signal at their respective inputs (sink points). As discussed above, the arrival times of the clock signals at the various sink points vary. The degree of variation in the arrival times at the two sink points depends on the variations (in physical and electrical characteristics) of the paths along which the clock signals propagate to reach the two sink points. For example, the difference in arrival times at sink points 330.1 and 330.2 may be 100 picoseconds 350. However, the difference in arrival times at sink points 330.2 and 330.3 may be as little as 20 picoseconds due to the proximity of these sink points to each other. Thus, a design in which the computation is split into computational components that are performed locally within each cascaded logic unit can reduce the clock skew that each logic unit must tolerate.

[0017] As mentioned above, during design, some time from the clock period must be allocated for the calculation 256 and some time must be reserved for the expected maximum clock skew 258. Therefore, the clock delay associated with the logic unit must be kept below the relied upon maximum clock skew. To ensure proper performance of the cascaded logic unit 340, the clock delay (or clock skew) caused by the different arrival times of the clock signals at the respective sink points associated with the interfaces between the logic units across the chip must be controlled. Furthermore, when a logic unit interfaces with another logic unit integrated on another chip 360, the clock signals arriving at the sink points associated with the interfaces between the logic units must be aligned (synchronized). That is, the time delay associated with the chip-to-chip interface 360 ​​must also be controlled, as will be further explained with reference to FIG. 4.

[0018] FIG. 4 is a schematic diagram of a multi-chiplet system 400 in which one or more features of the present disclosure may be implemented. The system 400 includes multiple chiplets 430.1-430.16 packaged in a 4×4 array. A clock generator 410 and a microcontroller 420 are integrated in a first chiplet 430.1, while logic circuits (e.g., logic units 220.1-220.3, 340 described with reference to FIGS. 2-3) are integrated in the other chiplets 430.2-430.16. A clock signal from the clock generator 410 is distributed to a clock unit 480 located at the center of each chiplet via wires that may be integrated in a carrier (e.g., packaging such as an interposer, silicon layer, or organic substrate) that holds the chiplets. The clock unit includes a clock receiver that receives the clock signal and a delay unit that can introduce a programmable delay offset into the received clock signal (as will be further described with reference to FIG. 5). Each clock unit 480 further distributes the received (and potentially delayed) clock signal via its chiplet's clock distribution tree 440, which propagates the clock signal to various sink points (e.g., points at interfaces 460, 470 between logical units).

[0019] Chiplet is relatively small (e.g., 5 square millimeters), therefore the difference in arrival time of clock signals at two sink points within a chiplet (i.e., clock skew) may be within 20 picoseconds. However, when logic units interface across chiplets, the difference in arrival time of clock signals at two adjacent sink points from different chiplets may be larger. As shown in FIG. 4, a pair of phase detectors 450 is placed at the interface between the pair of chiplets. The phase detectors 450 are configured to measure the phase (delay) between two clock signals, as will be further described with reference to FIG. 5. For example, one logic unit integrated in an area within chiplet 430.2 may interface with another logic unit integrated in an adjacent area within another chiplet 430.3. The output of one logic unit is the input of the other logic unit, so the latter needs to be signaled when the former has finished its operation. To that end, as described above, the clock signals measured at the respective sink points 460, 470 of the logic units should be aligned (synchronized). Such alignment may be done based on the phase between these clock signals, which may be measured by a pair of phase detectors 450.1, as disclosed herein. Thus, each phase detector 450 in system 400 provides a phase measurement indicative of the current clock skew that exists between clock signals arriving at two sink points across adjacent chiplets. Embodiments disclosed herein utilize such phase measurements to reduce clock skew by adjusting the delay between the clock signals distributed to the two sink points.

[0020] Microcontroller 420 is configured to dynamically adjust the delay of the clock signal distributed across each of chiplets 430.2-430.16. Microcontroller 420 is communicatively linked to phase detector 450 and to programmable delay units (e.g., delay units 542, 552 shown in FIG. 5) in clock unit 480. Due to manufacturing variations, different chiplets may respond differently to changes in temperature and / or voltage levels. Furthermore, due to variations in the paths along which the clock signal travels to its respective sink point, the clock signal has different arrival times. The difference in arrival times for two sink points that are close together within a chiplet may be tolerable (e.g., less than 20 picoseconds). However, the difference in arrival times for two sink points located on two different chiplets may be large. Thus, clock skew caused by differences in signal propagation speeds in different distribution paths across and within each chiplet needs to be reduced by introducing delays at different points on these distribution paths as disclosed herein.

[0021] Microcontroller 420 is configured to monitor phase measurements received from phase detectors 450. Phase detectors 450 may be integrated into a carrier that holds the chiplets (or may be integrated on the interface edge of the chiplets themselves), where each pair of phase detectors (e.g., 450.1) is located at an interface location between the chiplets (e.g., by sink points 460, 470 where two logic units from chiplet 430.2 and chiplet 430.3 interface). If the phase measurement received from either of the phase detectors associated with a pair of chiplets exceeds a tolerance (maximum clock skew), microcontroller 420 can instruct clock unit 480 of the chiplet with the faster clock signal (from that pair) to adjust its programmable delay so that the skew is reduced. For example, if it is determined based on phase measurements received from a pair of phase detectors 450.1 associated with two interface logic units (e.g., one logic unit at sink point 460 of chiplet 430.2 and another logic unit at sink point 470 of chiplet 430.3) that the clock signal arriving at sink point 460 leads the clock signal arriving at sink point 470, microcontroller 420 can instruct the clock unit of chiplet 430.2 to introduce a delay in the clock signal distributed throughout chiplet 430.2 (or to introduce a delay in the clock signal distributed towards sink point 460), thereby reducing the skew (as measured by phase detector 450.1) between the pair of chiplets.

[0022] 5 is a block diagram of an example system 500 for clock delay compensation in which one or more features of the present disclosure may be implemented. The system 500 illustrates clock delay compensation for two paths in which a clock signal generated by a clock generator 530 (e.g., 410) is distributed to two adjacent chiplets, namely, chiplet A 540 and chiplet B 550. As shown, associated with chiplet A 540 is a clock receiver 541 and a programmable delay unit 542 (part of chiplet A's clock unit 480), as well as a clock distribution path 544 that terminates at a sink point 545. Similarly, associated with chiplet B 550 is a clock receiver 551 and a programmable delay unit 552 (part of chiplet B's clock unit 480), as well as a clock distribution path 554 that terminates at a sink point 555. These sink points 545, 555 provide the clock signal (distributed via the respective distribution paths 544, 554) to the respective logic units in chiplet A and chiplet B that interface with each other. Therefore, the clock signals measured at these sink points need to be aligned. First, chiplet A is configured to receive 541 a clock signal and then delay 542 the clock signal by an initial delay offset (e.g., equal to zero). Similarly, chiplet B is configured to receive 551 a clock signal and then delay 552 the clock signal by an initial delay offset (e.g., equal to zero). As explained, due to manufacturing variations in the circuits and wires through which the clock signal propagates, the clock signal generated by clock generator 530 is split into clock signals that arrive at sink points 545, 555 of chiplet A and chiplet B at different times. This difference in arrival times is measured by a pair of phase detectors 546, 556, as explained below.

[0023] Phase detectors 546 and 556 associated with chiplet A and chiplet B, respectively, are integrated into the carrier carrying the chiplets. Each phase detector is configured to measure the phase between the clock signal of its associated chiplet and the clock signal provided by the adjacent chiplet. However, to ensure that the phase measurements are accurate, the paths that the signals travel to reach the inputs of each phase detector need to be of the same length. For example, phase detector 546 is configured to measure the phase between a clock signal arriving at sink point 545 and a clock signal arriving at sink point 555. To ensure that this phase measurement is accurate, the paths that the signals travel from their respective sinks along the inputs of the phase detector need to be equal in length. That is, the path from sink point 545 to input 545.2 needs to be equal to the path from sink point 555 to input 555.1. Similarly, phase detector 556 is configured to measure the phase between a clock signal arriving at sink point 555 and a clock signal arriving at sink point 545. To ensure an accurate measurement of this phase, the path from sink point 555 to input 555.2 should be equal in length to the path from sink point 545 to input 545.1. The resulting phase measurements 548 and 558 of phase detectors 546 and 556, respectively, can be accessed by microcontroller 510 (e.g., microcontroller 420 of FIG. 4).

[0024] Thus, microcontrollers 510, 420 are configured to monitor the phases measured by phase detectors 450 associated with pairs of chiplets across system 400. The microcontrollers can access registers of the phase detectors that store the phase measurements via control bus 520. Based on the phase measurements received from the phase detectors associated with a pair of chiplets, e.g., 546 and 556, the microcontroller can determine whether one of the chiplets, e.g., 540 or 550, is lagging (slow) relative to the other. The microcontroller can then adjust the delay applied by the delay unit, e.g., 542 or 552, of the faster chiplet. For example, if it is determined that the clock signal at sink point 545 of chiplet A is lagging the clock signal at sink point 555 of chiplet B, microcontroller 510 can update the delay offset stored in the register of delay unit 552 via control bus 520. In response, delay unit 552 will adjust the current delay applied to the clock signal propagating through distribution path 554 based on its updated delay offset, i.e., delay unit 552 effectively slows down the clock signal so that when it reaches sink point 555 it will be aligned with the clock signal arriving at sink point 545.

[0025] Thus, through analysis of the phase measurements associated with the interface between the pair of chiplets, microcontroller 420 can adjust the delay applied by the delay unit in clock unit 480 of the pair of chiplets, as described above. For example, as a result of analysis of the phase measurements received from detectors 450.1 and 450.3 associated with the interface between the pair of chiplets 430.2 and 430.3, microcontroller 420 can update the delay offset stored in the register of the delay unit of chiplet 430.2 based on which the delay unit will delay the clock signal through the distribution tree of chiplet 430.2. Similarly, as a result of analysis of the phase measurements received from detectors 450.2 and 450.4 associated with the interface between the pair of chiplets 430.3 and 430.4, microcontroller 420 can update the delay offset stored in the register of the delay unit of chiplet 430.4 based on which the delay unit will delay the clock signal through the distribution tree of chiplet 430.4. In this manner, the microcontroller 420 may be configured to periodically read the phase measurement from the phase detector 450 and adjust the delay applied by the delay unit within the clock unit 480 .

[0026] In one aspect, based on the phase measurements, microcontroller 420 may determine a global delay offset for chiplet 430 that results in a global propagation delay in the clock signal distributed across the chiplet. However, in addition to or as an alternative to the global delay, microcontroller 420 may determine multiple local delays for the chiplet that result in different local propagation delays in the clock signal distributed across the chiplet. That is, the delay units of the chiplet may delay the clock signal distributed along different paths of the distribution tree according to different delay offsets. For example, microcontroller 420 may determine a global delay offset to be applied to the clock signal received by the clock unit of chiplet 430.3 based on the phase measurements received from phase detectors 450.1-450.4 associated with chiplet 430.3, and / or may determine four different local delay offsets to be applied to the clock signal distributed via different paths branching off from the distribution tree. Thus, receivers 541, 551 and delay units 542, 552 of a clock unit 480 associated with a chiplet are configured to receive a clock signal from clock generator 410, split the received signal into multiple clock signals, and apply different delay offsets to each clock signal before propagating the different delayed signals through respective paths of the chiplet's clock distribution tree.

[0027] In one aspect, based on the phase measurements associated with a pair of chiplets, the microcontroller 420 can determine which one of the chiplets is faster and adjust the respective delay offset to effectively slow down the propagation of the clock signal through that chiplet. Iterating through the pair of chiplets 430.2-430.16, the microcontroller can align the propagation of the clock signals in the different chiplets by slowing the propagation of those clock signals to match the propagation of the clock signal in the slowest chiplet.

[0028] FIG. 6 is a flow chart of an example method 600 for clock delay compensation, based on which one or more features of the present disclosure may be implemented. The method 600 adjusts a delay applied to a clock signal propagating through a distribution tree across chiplets of the system 400. In step 610 of the method 600, a clock signal is distributed across the clock distribution tree of each chiplet by a clock generator. In step 620, a phase is measured by a phase detector. Each measured phase is associated with a chiplet and indicates a propagation speed of the clock signal across the distribution tree of that chiplet. Then, for each chiplet, the method 600 performs steps 630 and 640. In step 630, a delay offset is determined by the microcontroller based on the phase measurement associated with the chiplet. In step 640, the propagation of the clock signal across the distribution tree of the chiplet is delayed based on the delay offset. In one aspect, delaying the clock signal propagation within the chiplet may also be performed by applying a local delay offset. Thus, for each chiplet, a local delay offset may be determined by the microcontroller based on the phase measurement associated with the chiplet. And then, based on the determined local delay offset, propagation of the clock signal across each path of the chiplet's distribution tree may be delayed using the delay unit associated with the chiplet.

[0029] Method 600 may be applied to periodically adjust clock skew throughout system 400. For example, phase measurements associated with pairs of chiplets may be compared. Based on the comparison, the slowest chiplet may be determined. The slowest chiplet is the chiplet whose clock signal propagation across its tree distribution is slower than the clock signal propagation across the distribution trees of all other chiplets. Once the slowest chiplet of system 400 is known, delay offsets of all other chiplets may be set such that the propagation speeds of all other chiplets match the propagation speed of the slowest chiplet. Performing this process periodically allows compensation of delays introduced into system 400 as environmental conditions (e.g., voltage and / or temperature) change. In one aspect, once the slowest chiplet is determined, further periodic compensation may be handed off to a hardware state machine (or may still be performed by a microcontroller, as described above).

[0030] It should be understood that many variations are possible based on the disclosure herein, and although features and elements have been described above in particular combinations, each feature or element can be used alone without the other features and elements, or in various combinations with or without the other features and elements.

[0031] The provided methods may be implemented in a general purpose computer, processor, or processor core. Suitable processors include, by way of example, general purpose processors, special purpose processors, conventional processors, digital signal processors (DSPs), multiple microprocessors, one or more microprocessors associated with a DSP core, controllers, microcontrollers, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Array (FPGA) circuits, any other type of integrated circuit (IC), and / or state machines. Such processors may be manufactured by configuring a manufacturing process using results of processed hardware description language (HDL) instructions and other intermediate data including netlists (such as instructions that may be stored on a computer readable medium). The results of such processing may be mask works, which are used in a semiconductor manufacturing process to manufacture processors implementing aspects of the embodiments.

[0032] The methods or flow charts provided herein may be implemented in a computer program, software or firmware embodied in a non-transitory computer-readable storage medium for execution by a general purpose computer or processor. Examples of non-transitory computer-readable storage media include read only memory (ROM), random-access memory (RAM), registers, cache memory, semiconductor memory devices, internal hard drives and disks, magnetic media such as removable disks, magneto-optical media, or optical media such as CD-ROM disks and digital versatile disks (DVDs).

Claims

1. 1. A method for clock delay compensation in a multi-chiplet system, comprising: a clock generator distributing a clock signal across a distribution tree of each chiplet; a phase detector measuring phase, each phase measurement being associated with one of the chiplets and indicative of a propagation speed of a clock signal through a distribution tree of the chiplet; For each chiplet, A microcontroller determining a delay offset based on a phase measurement associated with the chiplet; and delaying propagation of a clock signal through a distribution tree of the chiplet using a delay unit associated with the chiplet based on the delay offset. method.

2. For each chiplet, The microcontroller determining a local delay offset based on a phase measurement associated with the chiplet; and delaying propagation of a clock signal through each path of a distribution tree of the chiplet using a delay unit associated with the chiplet based on the local delay offset.

2. The method of claim 1.

3. Each phase measurement associated with a chiplet indicates a difference between an arrival time of a clock signal at a first sink point that serves a first logic unit in the chiplet and an arrival time of a clock signal at a second sink point that serves a second logic unit in another chiplet.

2. The method of claim 1.

4. the first logic unit interfaces with the second logic unit, and the clock signal controls the interface. The method of claim 3.

5. a path from the first sink point to an input of one of the phase detectors is extended to match a length of a path from the second sink point to another input of the phase detector; The method of claim 3.

6. comparing phase measurements associated with any pair of the chiplets; determining a slowest chiplet based on the comparison, wherein propagation of a clock signal through a tree distribution of the slowest chiplet is slower than propagation of a clock signal through a tree distribution of all other chiplets.

2. The method of claim 1.

7. Determining the delay offset for each chiplet includes: setting the delay offset such that a propagation speed of a clock signal through the distribution tree of the chiplet matches a propagation speed of a clock signal through the distribution tree of the determined slowest chiplet. The method of claim 6.

8. 1. A multi-chiplet system for clock delay compensation, comprising: At least one processor; A memory for storing instructions, The instructions, when executed by the at least one processor, a clock generator distributing a clock signal across a distribution tree of each chiplet; a phase detector measuring phase, each phase measurement being associated with one of the chiplets and indicative of a propagation speed of a clock signal through a distribution tree of the chiplet; For each chiplet, A microcontroller determining a delay offset based on a phase measurement associated with the chiplet; delaying propagation of a clock signal through a distribution tree of the chiplet using a delay unit associated with the chiplet based on the delay offset; causing the multi-chiplet system to Multi-chiplet system.

9. The instruction: For each chiplet, The microcontroller determining a local delay offset based on a phase measurement associated with the chiplet; delaying propagation of a clock signal through each path of a distribution tree of the chiplet using a delay unit associated with the chiplet based on the local delay offset; causing the multi-chiplet system to The multi-chiplet system of claim 8.

10. Each phase measurement associated with a chiplet indicates a difference between an arrival time of a clock signal at a first sink point that serves a first logic unit in the chiplet and an arrival time of a clock signal at a second sink point that serves a second logic unit in another chiplet. The multi-chiplet system of claim 8.

11. the first logic unit interfaces with the second logic unit, and the clock signal controls the interface. The multi-chiplet system of claim 10.

12. a path from the first sink point to an input of one of the phase detectors is extended to match a length of a path from the second sink point to another input of the phase detector; The multi-chiplet system of claim 10.

13. The instruction: comparing phase measurements associated with any pair of the chiplets; determining a slowest chiplet based on the comparison, wherein propagation of a clock signal through a tree distribution of the slowest chiplet is slower than propagation of a clock signal through a tree distribution of all other chiplets; causing the chiplet system to The chiplet system of claim 8.

14. Determining the delay offset for each chiplet includes: setting the delay offset such that a propagation speed of a clock signal through the distribution tree of the chiplet matches a propagation speed of a clock signal through the distribution tree of the determined slowest chiplet. The system of claim 13.

15. 1. A computer-readable storage medium comprising instructions executable by at least one processor for implementing a method for clock delay compensation in a multi-chiplet system, the method comprising: The method comprises: a clock generator distributing a clock signal across a distribution tree of each chiplet; a phase detector measuring phase, each phase measurement being associated with one of the chiplets and indicative of a propagation speed of a clock signal through a distribution tree of the chiplet; For each chiplet, A microcontroller determining a delay offset based on a phase measurement associated with the chiplet; and delaying propagation of a clock signal through a distribution tree of the chiplet using a delay unit associated with the chiplet based on the delay offset. A computer-readable storage medium.

16. For each chiplet, The microcontroller determining a local delay offset based on a phase measurement associated with the chiplet; and delaying propagation of a clock signal through each path of a distribution tree of the chiplet using a delay unit associated with the chiplet based on the local delay offset.

16. The computer readable storage medium of claim 15.

17. Each phase measurement associated with a chiplet indicates a difference between an arrival time of a clock signal at a first sink point that serves a first logic unit in the chiplet and an arrival time of a clock signal at a second sink point that serves a second logic unit in another chiplet.

16. The computer readable storage medium of claim 15.

18. the first logic unit interfaces with the second logic unit, and the clock signal controls the interface.

20. The computer readable storage medium of claim 17.

19. a path from the first sink point to an input of one of the phase detectors is extended to match a length of a path from the second sink point to another input of the phase detector; 20. The computer readable storage medium of claim 17.

20. comparing phase measurements associated with any pair of the chiplets; determining a slowest chiplet based on the comparison, wherein propagation of a clock signal through a tree distribution of the slowest chiplet is slower than propagation of a clock signal through a tree distribution of all other chiplets.

16. The computer readable storage medium of claim 15.

Images