TECHNIQUES FOR RESONANT ROTATIONAL TECHNIQUES FOR THE-TO-THE-COMMUNICATION

Abstract

Various embodiments provide devices, systems, and methods for resonant rotary clocking for die-to-die (D2D) communication in a multi-die system. A base die can include a resonant ring structure to form a plurality of traveling-wave rotary oscillators (RTWOs) coupled together in a rotary oscillator array (ROA). The ROA can provide synchronized clock signals at deterministic phase points tapped from the resonant ring structure. Multiple dies can be coupled to the base die and can receive the tapped clock signals from their respective tap points. The clock signals can be used for die-to-die communication and / or other purposes. Further embodiments can be described and claimed.

Description

Area

[0001] Embodiments of the present invention generally relate to the technical field of electronic circuits and, more particularly, to resonant rotary clocking in electronic circuits. State of the art

[0002] The background description provided herein is intended to generally set forth the context of the disclosure. Work by the inventors identified herein, to the extent described in this "Prior Art" section, as well as aspects of the description not otherwise considered prior art at the time of filing, are not expressly or implicitly admitted as prior art against the present disclosure. Unless otherwise noted herein, the approaches described in this section are not prior art to the claims in this disclosure and are not admitted as prior art by inclusion in this section.

[0003] The silicon industry is moving toward die decomposition and chiplet-based systems, in which smaller, heterogeneous dies are integrated on a single substrate, enabling superior functionality and improved operating characteristics. Designing a robust, fast, low-skew, low-jitter, and low-power clock across such chiplet-based systems is extremely challenging. The traditional globally asynchronous locally synchronous (GALS) solution presents several challenges related to overhead and design verification, which developers of asynchronous solutions have generally avoided. However, enabling clock synchronization for a chiplet-based system (across multiple dies) is extremely difficult and remains a key challenge in multi-die systems. Short description of the drawings

[0004] Embodiments will be readily understood from the following detailed description taken in conjunction with the accompanying drawings. For convenience of this description, like reference numerals designate like structural elements. Embodiments are illustrated in the figures of the accompanying drawings by way of example and not by way of limitation. Fig. 1A illustrates a ring structure for a traveling wave rotary oscillator (RTWO) according to various embodiments. Fig. 1B illustrates a rotary oscillator array (ROA) including multiple coupled ring structures, according to various embodiments. Fig. 2A illustrates a multi-die system having multiple dies (e.g., chiplets) coupled to a base die, according to various embodiments, wherein the base die includes resonant rings of an ROA. Fig. Figure 2B illustrates a first example implementation including an active base die, with the inverters implemented in the base die. Fig. Figure 2C illustrates a second example implementation including a passive base die, where the inverters are implemented in the chiplets and coupled to the spinning rings (e.g., via microbumps). Fig. 3A illustrates a multi-die system (e.g., system-in-package (SIP)) with resonant RTWO rings in a base die according to various embodiments. Fig. 3B illustrates exemplary D2D I / O circuitry of the first and second dies in a multi-die system according to various embodiments. Fig. 4 illustrates an example of a rotating ring structure and associated tapping points across a first die and a second die for a scheme in which the clock signal is tapped at equal phase points, according to various embodiments. Fig. 5 illustrates example waveforms associated with die-to-die communications with resonant clocks having similar phase points, according to various embodiments. Fig. 6 illustrates an example of a rotating ring structure and associated tapping points across a first die and a second die for a multi-phase tapping scheme according to various embodiments. Fig. 7 shows resonant clock signals with different phases to illustrate a transmission window for the multi-phase tapping scheme according to various embodiments. Fig. 8 illustrates example waveforms associated with die-to-die communications with a 3x pump ratio, according to various embodiments. Fig. Figure 9 illustrates a multi-die system with a first die and a second die coupled to an interposer. A rectangular resonant ring is included in the interposer, extending across the region between the two dies, according to various embodiments. Fig. 10 illustrates an exemplary rectangular ring structure with phase points according to various embodiments. Fig. 11 illustrates example waveforms associated with die-to-die communications, according to various embodiments. Fig. 12 illustrates a custom rotary oscillator array (CROA) according to various embodiments. Fig. 13 illustrates an example scheme for die-to-die communication using a custom rotary oscillator according to various embodiments. Fig. 14, Fig. 15, Fig. 16 and Fig. 17 illustrate example schemes for die-to-die communication using a custom rotary oscillator array according to various embodiments. Fig. 18A illustrates an exemplary structure of a square ring traveling wave rotary oscillator according to various embodiments. Fig. 18B illustrates an exemplary structure of a square ring standing wave rotary oscillator according to various embodiments. Fig. 19 illustrates an example scheme for die-to-die communication using a rotating wave oscillator having switches for switching between a standing wave mode and a traveling wave mode, according to various embodiments. Fig. 20 illustrates an example scheme for die-to-die communication using a custom rotating wave oscillator switchable between a standing wave mode and a traveling wave mode, according to various embodiments. Fig. 21 illustrates a rotary oscillator array with several of the rotary oscillators from Fig. 19, which are coupled (e.g., short-circuited) to each other, according to various embodiments. Fig. 22 illustrates an example system configured to employ the devices and methods described herein according to various embodiments. Detailed description

[0005] Various embodiments herein provide devices, systems, and methods for resonant rotary clocking for die-to-die communication according to various embodiments. For example, a multi-die system may include an interposer and two or more dies coupled to the interposer. The interposer may include a resonant rotary ring structure to form one or more resonant rotary oscillators (e.g., of a resonant rotary oscillator array). The resonant rotary oscillators may be traveling-wave and / or standing-wave oscillators. The dies may tap respective clock signals from the rotary ring structure and use the clock signals for die-to-die communication.

[0006] In the following detailed description, reference is made to the accompanying drawings, which form a part hereof, wherein like reference characters designate like parts throughout, and in which is shown by way of illustration embodiments which may be practiced. It is understood that other embodiments may be utilized and structural or logical changes may be made without departing from the scope of the present disclosure. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the embodiments is defined by the appended claims and their equivalents.

[0007] Various operations may be described as multiple discrete actions or operations performed sequentially in a manner that is extremely helpful for understanding the claimed subject matter. However, the order of description should not be construed to imply that these operations necessarily depend on that order. In particular, these operations need not be performed in the order presented. The described operations may be performed in a different order than that of the described embodiment. In additional embodiments, various additional operations may be performed and / or described operations may be omitted.

[0008] The terms "substantially," "near," "approximately," "close to," and "about" generally refer to being within + / - 10% of a target value. Unless otherwise specified, the use of the ordinal adjectives "first," "second," and "third," etc., to describe a common object merely indicates that reference is being made to different instances of like objects, and is not intended to imply that the objects so described must be in any given sequence, whether temporally, spatially, in order of precedence, or in any other way.

[0009] For the purposes of this disclosure, the phrases "A and / or B" and "A or B" mean (A), (B), or (A and B). For the purposes of this disclosure, the phrase "A, B and / or C" means (A), (B), (C), (A and B), (A and C), (B and C), or (A, B and C).

[0010] The description may use the phrases "in one embodiment" or "in embodiments," each referring to one or more of the same or different embodiments. Furthermore, the terms "comprising," "including," "having," and the like, as used with respect to embodiments of the present disclosure, are synonymous.

[0011] As used herein, the term "circuitry" may refer to, be a part of, or include an application-specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group), a combinational logic circuit, and / or other suitable hardware components that provide the described functionalities. As used herein, "computer-implemented method" may refer to any method performed by one or more processors, a computer system having one or more processors, a mobile device such as a smartphone (which may include one or more processors), a tablet, a laptop computer, a set-top box, a game console, and so on.

[0012] Traveling-wave rotary oscillators (RTWOs) can incorporate a ring structure on which the clock signal travels as a traveling wave. Multiple RTWOs can be coupled together in a rotary oscillator array (ROA) to distribute the clock signal over a larger area. For example, Fig. 1A illustrates an RTWO including rotating rings 102a and 102b. The rotating rings 102a-b may be coupled together so that the clock signal can move continuously along both rotating rings 102a-b. The clock signal may be tapped at different tap points on the ring structure to provide different phases of the clock signal, as shown (e.g., 0°, 45°, 90°, etc.). The rotating rings 102a-b may be implemented using interconnects (ICs) and / or other suitable conductive structures for the transmission lines. The RTWO 100 may further include one or more pairs of inverters 104a-b coupled in anti-parallel between the rotating rings 102a-b to adiabatically feed and amplify the signals. In some embodiments, the pairs of inverters 104a-b may be complementary metal-oxide-semiconductor (CMOS) inverters, although other types of inverters / transistors may also be used.Additionally or alternatively, the pairs of inverters 104a-b and / or may be evenly distributed along the transmission lines.

[0013] In embodiments, the RTWO may be modeled as an inductor-capacitor (LC) oscillator, where the frequency f osc is estimated by: ƒosc≈vp2l=1LTCT

[0014] In equation (1) v p is the phase velocity, and l is the length / circumference of the ring. The factor 2 (in the denominator) results from the fact that the pulse requires two complete rounds for a single cycle. Furthermore, the total inductance and total capacitance of a rotating ring are given by L T or C T The total inductance L T depends on the geometry of the rotating ring and C T is the total capacity of the ring, the intermediate connections and devices connected to the rotating ring.

[0015] Fig. 1B illustrates an exemplary ROA 150 that includes multiple RTWOs 100 coupled together. The RTWOs may be shorted to each other at shorting locations 152ab. For example, the corner of the outer ring of a first RTWO (e.g., ring 2 in Fig. 1B) at the corner of the inner ring of a second RTWO (e.g. Ring 3 in Fig. 1B) may be shorted at shorting point 152a, and the corresponding corner of the inner ring of the first RTWO may be shorted to the corresponding corner of the outer ring of the second RTWO at shorting point 152b. The short may enable the ROA 150 to provide a clock signal at synchronized tap points via the ROA 150. Other configurations of the RTWOs may also be used according to various embodiments. Furthermore, multiple RTWOs and / or ROA structures may be coupled together to distribute the clock signals across a reticle area.

[0016] Various embodiments herein include techniques for using ROAs to provide clock synchronization across a multi-die system (MDS) for die-to-die (D2D) communication (e.g., D2D input-output (IO)). The MDS may, for example, include a system-in-package (SIP). The MDS may include multiple dies coupled to a common base die (e.g., interposer) and / or otherwise integrated into a same package. The dies may include heterogeneous dies of different types and / or capabilities. Additionally or alternatively, the dies may include multiple similar / same dies. For example, the dies may include one or more processor dies, memory dies, graphics processor dies, input-output (I / O) dies, power management dies, and / or other suitable die types.

[0017] Aspects of various embodiments herein may include, but are not limited to: - Use the resonant clock as a common I / O clock for D2D I / O communication, e.g., by sampling the clock signal at deterministic phase points. This can eliminate the need for I / O clock infrastructure found in previous designs, such as phase-locked loops (PLLs), strobe-passing drivers, and delay-locked loops (DLLs). In some embodiments, the clock signal can be sampled at the same phase point on both the transmit and receive sides. In other embodiments, the receive-side clock signal can be sampled at a different phase point than the transmit-side clock signal. For example, the receive-side clock signal can be 45-135 degrees ahead in phase of the transmit-side clock signal. In some embodiments, the receive-side phase lead can improve setup margin and / or enable faster operation. - The overall D2D circuit design is simplified, e.g., by replacing strobe generation / recovery circuitry (e.g., PLLs, strobe drivers, DLLs) with inverter pairs, while also reducing area and power. - A rectangular resonant ring may be included on the interposer, with longer edges overlapping the periphery of the upper dies coupled to the interposer. - Custom rotary ring structures for D2D communication. The custom rotary ring structures can be implemented using an active and / or passive interposer. The ROA for D2D IO can be a traveling wave oscillator and / or a standing wave oscillator. Some embodiments can include a multi-mode oscillator circuit switchable between a traveling wave mode and a standing wave mode. - These and other aspects of various embodiments are described below. Implementation using a passive or active interposer

[0018] In various embodiments, the resonant clock circuit may be implemented in a multi-die system using a passive or active interposer (also referred to as a base die). Fig. 2 illustrates an exemplary multi-die system 200 including a plurality of dies (e.g., chiplets) 202 coupled to a base die 204 (e.g., via µ-bumps 206 and / or another suitable mechanism). The base die 204 may include resonant rings 208 formed therein, e.g., in one or more metal layers. The clock signals on the resonant rings 208 may be tapped (e.g., from respective tap points) and provided to the dies 202 through the µ-bumps, e.g., as reference signals for synchronization. Due to the nature of the ROAs, multiple tap points exist on the resonant ring structure that we may use for synchronization, as discussed further herein.

[0019] In some embodiments, the multi-die system 200 may include an active base die 204. For example, Fig. 2B illustrates an active base die 204 including inverter pairs 210 implemented in the base die 204 and coupled between the inner and outer rings of the resonant rings 208. Fig. 2C illustrates an example of a passive base die 204 in which the inverter pairs 210 are implemented within another die 212. The inverter pairs 210 may be coupled to the resonant rings 208 via μ-bumps and / or another suitable mechanism. In some embodiments, the die 212 may correspond to the dies 202 of the multi-die system 200 (e.g., each die 202 may include inverter pairs coupled to respective resonant rings 208 of the base die 204).

[0020] The resonant rings in base die 204 may enable dies 202 to tap synchronized clock signals with deterministic phase points. In some embodiments, base die 204 may include bumps 214 coupled to a bottom surface of base die 204, for example, to mount the multi-die system on a motherboard or other circuit structure. In some embodiments, bumps 214 may be larger (e.g., C4 bumps) than µ-bumps 206 used to couple die 202 to base die 204.

[0021] Silicon interposer-based systems enable the integration of heterogeneous dies, benefiting from yield and cost advantages. The interposer footprint is important because passive interposers demonstrate superior yield with cost reduction through die partitioning, while active interposers demonstrate superior performance but at the cost / yield trade-off. Embodiments herein allow the resonant clock circuit to be used with either a passive or active interposer. Alternative phase reception for die-to-die (D2D) input-output (I / O) circuits

[0022] In various embodiments, the synchronized resonant clock signal described herein may be used for die-to-die (D2D) communication (also referred to as D2D input-output (IO)), e.g., in a multi-die system. In some embodiments, D2D communication may use alternative phase tap points on the transmit (Tx) and receive (Rx) sides (e.g., on the Tx-side serializer and the Rx-side deserializer), as described further below.

[0023] Performance figures for D2D I / O include bandwidth (BW) / mm and energy / bit. In previous implementations, D2D I / O typically includes a phase-locked loop (PLL) on the Tx side to generate high-speed edges used to serialize data and a strobe burst (e.g., for higher BW / mm). This burst is forwarded to the receiver, where it is deserialized and transferred to the Rx die clock domain (e.g., using a clock-domain crossing (CDC) first-in / first-out (FIFO)).

[0024] This I / O clock infrastructure (e.g., PLLs on the Tx side, strobe forwarding drivers, and delay-locked loops (DLLs) on the Rx side) contributes a dominant chunk of the total D2D I / O energy / bit and occupies a significant amount of real estate. To improve energy / bit, schemes typically adopt approaches such as voltage scaling or balancing the data line / strobe ratio.

[0025] According to various embodiments herein, a rotary oscillator array may be disposed across the base die (e.g., interposer), providing deterministic phase points across the multi-die system (e.g., SIP). In embodiments, this resonant clock may be used as a common I / O clock for D2D communication. For example, the resonant clock signal may be tapped at deterministic phase points on the Tx and Rx sides of the D2D I / O within respective dies (e.g., chiplets). Using the synchronized resonant clock signal may eliminate the need for Tx-side PLLs, strobe forwarding, and Rx-side DLLs of previous techniques, thereby reducing the total energy / bit and / or the footprint of the D2D I / O.

[0026] In some embodiments, alternative phase tap points on the Tx side and the Rx side can be used for D2D I / O. For example, on the Tx side, data can be serialized using phase A of the resonant clock signal. On the Rx side, data can be acquired using phase B of the resonant clock. The acquired data can be deserialized and passed to the CDC FIFO. In one example, phase B leads phase A in phase, e.g., by 45-135 degrees. The phase conduction between phase B and phase A can be used to improve the setup margin of the I / O scheme (e.g., by 16-37%), enabling faster operation.

[0027] The D2D I / O scheme described herein can complement voltage scaling techniques. Accordingly, voltage scaling can also be used in some embodiments. Additionally, the absence of strobe forward lines means that more data lines can be included in the same circuit area.

[0028] Fig. 3A illustrates a multi-die system 300 (e.g., SIP) with a resonant oscillator array used for D2D communication, according to various embodiments. The multi-die system 300 includes multiple dies (e.g., chiplets) 302a-f coupled to a base die (e.g., interposer) 304. The base die 304 may include resonant rings 306 to form an ROA, as described herein. The left side of Fig. 3A illustrates the base die 304 without the dies 302a-f to illustrate the resonant rings 306. In some embodiments, the individual dies 302a-f may include one or more cross-coupled inverter pairs to excite the resonant rings 306 of the base die. Alternatively or additionally, the base die 304 may be an active die that includes cross-coupled inverter pairs to excite the resonant rings 306.

[0029] The dies 302a-f may further include respective I / O circuitry (e.g., PHY circuitry) to communicate with other dies of the dies 302a-f. For example, Fig. 3A illustrates an I / O circuit arrangement 308a of die 302a and an I / O circuit arrangement 308b of die 302b. Although in Fig. 3, the dies 302a-b may also include one or more I / O circuits for communicating with one or more of the other dies 302c-f. Additionally or alternatively, the other dies 302c-f may include I / O circuits for communicating with one or more of the other dies 302a-f.

[0030] In embodiments, I / O circuits 308a-b may communicate with each other (e.g., transmit and / or receive data) via a bus 310. Bus 310 may include one or more communication paths (e.g., data wires). For example, bus 310 may include multiple communication paths for parallel communication. In various embodiments, the resonant clock signals described herein may be used at dies 302a-b to serialize data (e.g., on the Tx side) and / or deserialize data (e.g., on the Rx side) transmitted on bus 310.

[0031] For example, Fig. 3B shows an example of I / O circuitry 308a and 308b according to some embodiments. As illustrated, I / O circuitry 308a may include logic 312, one or more serializers 314 coupled to the logic, and one or more Tx drivers 316 coupled to the serializers. Logic 312 may provide data to the one or more serializers 314. The one or more serializers 314 may receive a resonant clock 318a from resonant clock circuitry (e.g., resonant rings 306 of Fig. 3A) and serialize the data based on the tapped clock. The one or more serializers 314 may provide the serialized data to respective Tx drivers 316 for transmission over respective communication paths of bus 310.

[0032] The I / O circuitry 308b may include one or more Rx drivers 320 coupled to the bus 310, one or more deserializers 322 coupled to respective Rx drivers 320, and logic 324 coupled to the one or more deserializers 322. The one or more Rx drivers 320 may receive the serialized data and forward the serialized data to the respective deserializers 322. The deserializers 322 may receive a resonant clock 318b from resonant clock circuitry (e.g., resonant clocks 306 of Fig. 3A) and deserialize the data based on the sampled clock. Deserializers 322 can pass the deserialized data to logic 324 for further processing.

[0033] The I / O circuitry 308a-b is merely an example, and other configurations of the I / O circuitry may be used with resonant clock signals according to various embodiments herein. For example, some embodiments may not use serializers and deserializers. In such an example, the transmitter may transmit data directly based on a Tx clock. The Tx clock may be the resonant clock signal or a frequency-adjusted version of the resonant clock signal. For example, the resonant clock signal may be used as the global clock for the multi-die system, and the I / O circuitry may use transmit / receive clocks that have a lower frequency than the global clock.

[0034] In some embodiments, the resonant clock signal can be sampled at the same phase points on the Tx and Rx sides. Accordingly, the entire period of the clock signal can be used as the D2D transmission window (e.g., time of flight + setup margin).

[0035] Fig. Figure 4 schematically illustrates an example of the rotating ring structure and tap points for a communication scheme using equal phase points. As shown, a first die 402a and a second die 402b may each include a resonant ring structure 404a-b. Fig. 4 further shows the phase of the resonant clock signal at various points on the resonant ring structure 404a-b. In embodiments, the first and second dies 402a-b may tap the resonant clock signal at corresponding tap points 406a-b corresponding to the same phase points. For example, the first and second dies 402a-b may include a serializer and / or deserializer for communications between the first and second dies 402a-b and may use the tapped clock signal to serialize and / or deserialize the data. By tapping similar phase points and using them as the I / O clock for the next serializer / deserializer, a high-speed clock period may be used as a D2D transmission window.

[0036] For example, Fig. 5 an example of the resonant clocks with similar phase points, the serial Tx data from Die 1 to Die 2 and the incoming sampled Rx data to Die 2.

[0037] In other embodiments, a multi-phase tapping scheme may be used for D2D I / O. Fig. Figure 6 illustrates an example of rotating ring structures 604a-b and tap points 606a-b across a first die 602a and a second die 602b for a multi-phase tapping scheme. Instead of tapping similar phase points as discussed above, alternative phase points can be used for tap points 606a-b. This approach can reduce the distance over which the high-speed clock is transmitted and / or increase the transmission window of the D2D I / O.

[0038] In various embodiments, the tapped clock signal used on the Rx side may be in phase with the tapped clock signal used on the Tx side, e.g., by 45-135 degrees. For example, consider the second die 602b, which is connected to the first die 602a in Fig. 6. The resonant clock phase variation on the Tx side at the tap points 606b is between 0 and 45 degrees, while on the Rx side the phase variation at the tap points 606a is 135-90 degrees.

[0039] In various embodiments, transmitting between any two phase points on the same line (e.g., corresponding tap points 606a-b) increases the transmission window, e.g., by 1 / 8 to 3 / 8 of the total period. Fig. Figure 7 illustrates an example of transmission windows 702a-b for a 135-degree phase difference and a 45-degree phase difference, respectively. Additionally, the total distance over which the clock signals are transmitted within a chiplet may be L len reduce (from 2*L len in case of using the same phase points).

[0040] Fig. Figure 8 illustrates various waveforms to show an example of the Tx-side transfer from one die to another for a 3x pump ratio. For example, Fig. 8, a resonant clock signal with phase 0, a first system clock (system clock 1), and a second system clock (system clock 2). The first and second system clocks may have a period that is three times the period of the resonant clock signal. Additionally, the second system clock has a different phase than the first system clock (e.g., deviating by 1 / 2 the period of the resonant clock signal, which corresponds to a 60-degree difference from the first system clock). Fig. Figure 8 further illustrates an input signal to a Tx serializer on Die 1 and serial Tx data being transferred from Die 1 to Die 2. Furthermore, Fig. 8 an input signal to a Tx serializer on Die 2 and serial Tx data transmitted from Die 2 to Die 1.

[0041] In some embodiments, D2D I / O traffic between the two dies can be asynchronous (since the timing is determined by the phase relationship between the Tx-side clock and the I / O clock). On the Rx side, the data can be deserialized and passed to the CDC FIFO. Rectangular resonance ring over chiplets

[0042] Fig. Figure 9 illustrates a multi-die system 900 including a first die 902a (C1) and a second die 902b (C2) coupled to an interposer 904. The multi-die system 900 may be a heterogeneous system in some embodiments. The dies 902a and 902b may include logic and / or memory.

[0043] In embodiments, the interposer 904 may include a resonant ring 906 disposed over the region adjacent to the dies 902a and 902b. For example, a first long edge 908a of the resonant ring 906 may be partially or completely located below the first die 902a, and a second long edge 908b of the resonant ring 906 may be partially or completely located below the second die 902b.

[0044] The dies 902a-b may include a respective D2D PHY circuitry 910a-b. In some embodiments, the D2D PHY circuitry 910a-b may be located above the respective long edge 908a-b of the resonant ring 906. The D2D PHY circuitry 910a-b may include data serializers and / or deserializers for data transmission. In some embodiments, the D2D PHY circuitry 910a-b may further include inverter pairs to excite the resonant ring 906. The resonant ring 906 provides a common strobe with a deterministic phase at any tap point. This strobe can be tapped by the D2D PHY circuitry 910a-b from the nearest points on both the serializer and the deserializer to transition between parallel data to a serial bit stream.

[0045] For example, at die 902a, resonant ring 906 can be tapped to pull the high-speed I / O clock used to serialize data and transfer the serialized data to die 902b (e.g., via D2D interconnects 912). At die 902a, a local I / O clock copy can be tapped from the nearest point to the resonant ring and used to deserialize data. The deserialized data can be passed to a clock domain crossing (CDC) first-in, first-out (FIFO) circuit.

[0046] The deterministic phase difference between tap points on dies 902a-b (e.g., as described in the next section and elsewhere herein) ensures a reliable setup / hold margin for the data passing across dies 902a-b. Data transmission using a strobe generated by a rectangular ring

[0047] Typically, the edge of a die is much longer than the die-to-die spacing. Fig. Figure 10 illustrates an exemplary resonant ring 1000 with phase points at different tap points of the specified resonant ring 1000. The comma separates the clock phases for the respective tracks that form the ring 1000 (e.g., the inner ring 1002a and the outer ring 1002b). For clarity, inverter pairs (which excite the ring 1000) are shown in Fig. 10 not shown.

[0048] In embodiments herein, an alternative track tapping scheme may be used. For example, if a signal is transmitted using the clock derived from outer ring 1002b, it is received using the clock derived from inner ring 1002a.

[0049] As an example, consider the I / O located near the center of the resonant ring 1000. On a first die (e.g., die 902a from Fig. 9), the clock can be tapped from the outer ring 1002b (e.g., at tap point 1004a) and used to serialize data. The tapped clock can have a phase of 45 degrees, as shown. At a second die (e.g., die 902b of Fig. 9) The clock may be tapped from the inner ring 1002a (e.g., at tap point 1004b) and used to deserialize the data. This tapped clock may have a phase of 315 degrees, as shown. This results in a total window of 270 degrees for transmission. For an 8G resonant ring, this translates to a transmission window of 93 ps. In some embodiments, the transmission window varies from P (e.g., at the bottom of the ring 1000) to half of P (e.g., at the top of the ring 1000), where P refers to one period of the I / O resonant clock.

[0050] Fig. Figure 11 illustrates several example waveforms to illustrate the alternative track tapping scheme according to various embodiments. For example, Fig. 11 shows the resonant clock signal at different phase points. Fig. 11 the input to the Tx serializer on the first die, where the resonant clock is used to serialize the data on the first die, where the serial data is transferred from the first die to the second die, and the resonant clock signal used to deserialize the data on the second die.

[0051] Serializers in the first die use the next resonant ring tap to transmit data. In Fig. Figure 11 shows a phase 45 as an example, which is used to serialize data on the TX side and transmit it to the RX side. On the RX side, the same tap outputs a 315-degree phase clock, which is used to deserialize data. D2D I / O using custom rotary ring structures

[0052] Various embodiments herein include custom rotary ring structures for a rotary oscillator array (e.g., for D2D I / O). The custom rotary rings may include on-chip interconnects and inverter pairs that are mobile terminated (as described herein) to generate a resonant clock signal with a 50% duty cycle. The custom rings and / or custom rotary oscillator arrays may be used to sample clock signals for D2D I / O. The resonant rings oscillate to generate an I / O clock with deterministic phase points across the dies. The I / O clock may be used to serialize and deserialize data.

[0053] For example, Fig. 12 illustrates an exemplary custom rotary oscillator array (CROA) 1200 according to various embodiments. The CROA includes a plurality of custom rotary oscillators 1202a-e coupled (e.g., shorted) together. The custom rotary oscillators 1202a-e are merely examples of various embodiments. The custom rotary oscillators 1202a-e may include any suitable shape, for example, to provide tap points at desired locations and / or clock signals with a desired phase at one or more locations. The custom rotary oscillators 1202a-e may have a rectilinear shape that is not rectangular. In some embodiments, one or more custom rotary oscillators 1202a-e may be combined with one or more regular (e.g., rectangular) rotary oscillators in a rotary oscillator array.

[0054] As with regular resonant rings, custom resonant rings can be implemented in the interposer (e.g., silicon interposer). In some embodiments, the inverter pairs for exciting the resonant ring can be implemented in the die coupled to the interposer and / or in the interposer itself. The inverter pairs can replace the conventional strobe infrastructure (e.g., PLLs, strobe drivers, DLLs) of previous I / O circuits.

[0055] The techniques described herein may enable the use of custom rotary rings, which can be used for D2D I / O in multi-die systems (e.g., heterogeneous multi-die systems). The custom rings may be coupled together to form custom rotary oscillator arrays to distribute the required clocks over a large area (e.g., the entire reticle). Embodiments may include a chiplet-aware resonant array implementation to identify the required clock tap points for D2D I / O. Accordingly, the shape of the resonant rings may be designed to provide tap points at desired locations to the top dies coupled to the interposer. The traveling-wave scheme provides a deterministic delay, which may facilitate use in D2D I / O circuits.This scheme can enable the use of either the same phase points on multiple user-defined rings and / or different phase points with deterministic delays on the user-defined rings for D2D I / O.

[0056] The resonant traveling wave scheme makes it possible to tap the clock signals from different points on the user-defined ring and provide them as inputs to the chiplets. Since the delay / phase at the tap points is deterministic, the difference in phase / delay is used as the transmission window. Fig. 13 illustrates an exemplary multi-die system 1300 with dies D1-D5 1302a-e coupled to an interposer 1304. The interposer 1304 may include one or more user-defined rotating rings 1306 (e.g., coupled to a rotary oscillator array). Each die D1-D5 may include one or more cross-coupled inverter pairs 1308 to excite the resonant rings 1306. The clock signals from the user-defined rotating ring 1306 may be provided to the dies D1-D5. For example, representative tap points are shown in Fig. 13 for dies D1 and D2. This principle can be extended to sample clocks from multiple points on the user-defined ring with deterministic delays. At any point on the ring, the clock delay 't', the clock phase 'θ', and the clock period 'T' are correlated by the following: θ360=tT

[0057] For example, for a 4GHz resonant ring in Fig. 13 For die D1, a clock can be tapped from a point on the ring, where θ=60°. For D2, the clock can be tapped from the same structure but from the inner ring, where θ=180°. This provides a window of 120°. This translates into the transmission window of 82 ps (e.g., using equation (2)). It should be noted that the clocks to different dies can additionally or alternatively be tapped from other / different points on the ring, e.g., depending on requirements.

[0058] Other schemes for D2D I / O using an array of user-defined rings may be used according to various embodiments, for example, illustrated Fig. 14 shows a multi-die system 1400 with dies 1402a-e on an interposer 1404. The interposer 1404 may include multiple ring structures 1406a-d coupled in an array (e.g., of 4 ring structures 1406a-d). The ring structures 1406a-d may be associated with the rotating ring 1306. Fig. 13. Since there is at least one identical phase point on each ring structure 1406a-d, the clocks for different dies 1402a-e can be tapped from these identical phase points. Furthermore, depending on the architecture and placement of the dies 1402a-e, the ring structures 1406a-e can be designed to enable a favorable transmit / receive window for D2D I / O. Thus, a chiplet placement-aware resonant rotary clock scheme can be implemented on the interposer 1404 for efficient D2D I / O.

[0059] Further examples of custom rotating array schemes can be found in the Fig. 15, Fig. 16 and Fig. 17 to demonstrate the applicability and use of custom rings for D2D I / O. Note that in each of these examples, the same phase points on multiple rings and / or different phase points with deterministic delays on the custom rings may be used for D2D I / O. It will be apparent that many other designs of custom rotary oscillators and / or oscillator arrays may be used according to various embodiments. Furthermore, resonant ring structures with different designs may be combined in a rotary oscillator array. D2D communication using resonant standing wave oscillators

[0060] In a traveling-wave rotating oscillator (RTWO), the clock signal continues to travel continuously until it encounters another wave along the medium or until it encounters a boundary with another medium. For an RTWO, distributed inverter pairs enable multiple phases. Rotating traveling waves can be implemented using square rings and / or custom rings, as described herein. Both square and custom rings can be distributed using array structures, as described herein. An exemplary RTWO 1800 with a square ring rotating structure is shown in Fig. 18A. The RTWO 1800 includes rotating rings 1802a-b coupled together via a Möbius junction and multiple pairs of cross-coupled inverters 1804a-b coupled between the rotating rings 1802a-b.

[0061] In a standing wave (SW) scheme using transmission lines, each point on the transmission line generates a sine wave of varying amplitude due to parasitic losses. The ring-based standing wave clock topology is motivated by the goal of combining the energy return feature of the rotary clock scheme with the constant phase (across all points in the ring) of the standing wave oscillator. The Möbius termination back to the source is used where the standing wave ring is a single cross-coupled rotary wave oscillator. An exemplary 1850 standing wave oscillator with a square ring standing wave structure is shown in Fig. 18B. The standing wave oscillator 1850 includes rotating rings 1852a-b coupled together via a Möbius junction, and a single pair of cross-coupled inverters 1854a-b coupled between the rotating rings 1852a-b. The standing wave oscillator 1850 may further include one or more clock recovery circuits 1856 coupled to the rings 1852a-b to generate an output clock based on the signals at the rings 1852a-b.

[0062] The effect of the Möbius connection at the cross-coupled inverter position is that the ring's clock information is biphasic. To obtain the required clock, a clock recovery circuit is used. It should be noted that due to the biphasic nature of the clock, the clock recovery circuits on one side must have their polarity reversed compared to those on the other side to allow in-phase tapping. In-phase and anti-phase waves meet at the center of this differential loop. A traveling wave, created due to conduction losses, finds its counterwave at this center and cancels the counterwave.

[0063] In the RTWO structure, due to the wave propagation in one direction along the transmission line, multiphase signals can be received from different positions on the transmission line. In the case of a standing wave oscillator (SWO), the generated signals have the same phase and different amplitudes. Both RTWO and SWO circuits share the same property of distributing a high-frequency clock with low skew and low jitter, which can be used for global clocking.

[0064] Various embodiments herein may use standing wave oscillators for D2D I / O. The standing wave oscillators may include rectangular (e.g., square or other rectangular) rings and / or custom (e.g., rectilinear) rings. The oscillators may include interconnects and inverter pairs (e.g., on-chip and / or interposer) that are terminated in a Möbius manner to generate a resonant clock signal. The embodiments may enable the use of a resonant ring with standing wave clocks that can be used for D2D I / O in any multi-die system.

[0065] As discussed above with respect to the traveling wave oscillator embodiments, the ring structures can be implemented in the interposer (e.g., silicon interposer). The inverter pairs can be implemented in the die coupled to the interposer and / or in the interposer itself. The ring oscillators can be stacked to form standing wave oscillator arrays to distribute the required clocks across the entire reticle. Embodiments can include a chiplet-aware resonant standing wave array implementation to identify the required clock taps for D2D EA.

[0066] One of the key properties of standing wave rings is that the clock phase is constant across the rings. However, the amplitude varies. A clock recovery circuit can be used to extract the square-wave clock. The clocks can be tapped from these structures using clock recovery circuits and provided across dies used to serialize and deserialize data. Thus, the standing wave rings enable constant-phase clocks. Accordingly, the clock signals can be tapped from various convenient points (e.g., with inherent synchronization / phase alignment by design) on the ring structures and provided as respective inputs to the dies (e.g., for D2D I / O).

[0067] Fig. 19 illustrates an exemplary multi-die system 1900 according to various embodiments. The multi-die system 1900 may include a plurality of dies 1902a-e coupled to an interposer 1904. The multi-die system 1900 may further include a rotary oscillator 1906 operable in a standing wave mode. The rotary oscillator 1906 may include rings 1908a-b with pairs of cross-coupled inverters 1910a-b coupled between the rings 1908a-b. The rotary oscillator 1906 may further include one or more switches 1912 coupled between the rings 1908a-b. The switches 1912 may be implemented in the interposer and / or the upper die(s). In some embodiments, the rotary oscillator 1906 may be switchable between a standing wave mode and a traveling wave mode. In traveling wave mode, all switches 1912 may be open.In standing wave mode, one of the switches 1912 may be closed to short the inner ring and the outer ring together. The circuit may include any suitable number of one or more switches coupled between the inner ring and the outer ring, such as four rings as in FIG. Fig. 19, or a different number of switches. In some embodiments, the rotary oscillator 1906 may operate only in the standing wave mode and not in the traveling wave mode. For example, one of the switches 1912 may be closed during operation.

[0068] The clock signals from the rotary oscillator 1906 may be provided to the dies 1902a-e. For example, Fig. 19 Clock signals tapped from rotary oscillator 1906 and provided to dies 1902a and 1902b via respective clock recovery circuits 1914a-b. Clock recovery circuits 1914a-b may generate a square wave clock signal from the signals received from rings 1908a-b. By tapping similar phase points and using them as the I / O clock for the nearest serializer / deserializer, the use of a high-speed clock period as the D2D transfer window is enabled. Note that I / O clocks may be slightly less than 2*L len (where L len the length of the ring) within the chiplet, since high-speed clocks can be started near the die boundaries (where TX / RX PHY circuits are located).

[0069] Fig. Figure 20 illustrates another exemplary multi-die system 2000 according to various embodiments. Multi-die system 2000 is similar to multi-die system 1900, except that rotary oscillator 2006 includes custom (e.g., rectilinear and non-rectangular) rings 2008a-b. The characteristics of the oscillations remain similar to those of the regular square-ring-based standing wave oscillator.

[0070] Fig. Figure 21 illustrates another multi-die system 2100 according to various embodiments. The multi-die system 2100 is similar to the multi-die system 1900, except that the multi-die system 2100 includes an array of oscillators 2106a-e that are coupled (e.g., shorted) together. The oscillators 2106a-e may be similar to the oscillator 1906 of Fig. 19. In other embodiments, the array may include different oscillator designs, such as one or more custom oscillators (e.g., oscillator 2006 and / or another type of custom oscillator). Since the phases on the oscillators 2106a-e are the same, the clocks for different dies 2102a-e may be tapped from any of the same phase points (through clock recovery circuits). For example, as shown in Fig. 21, die 2102a may receive a clock signal from oscillator 2106a via a clock recovery circuit 2114a, and die 2102b may receive a clock signal from oscillator 2106b via a clock recovery circuit 2114b.

[0071] Furthermore, the resonant rings can be implemented to enable a favorable transmit / receive window for D2D I / O, e.g., depending on the architecture and / or placement of the 2102a-e dies. Thus, a chiplet placement-aware resonant rotary clock scheme can be implemented on the interposer for efficient D2D I / O. It should be noted that this scheme can be extended to custom ring-based standing wave oscillator arrays and other array topologies, as discussed herein. Example system

[0072] Fig. Figure 22 illustrates an example of components that may be present in a computing system 3750 for implementing the techniques described herein. The computing system 3750 may include any combination of the hardware or logic components referenced herein. The components may be implemented as integrated circuits, portions thereof, discrete electronic devices or other modules, instruction sets, programmable logic or algorithms, hardware, hardware accelerators, software, firmware, or a combination thereof adapted therefor in the computing system 3750, or as components otherwise integrated into a chassis of a larger system. For one embodiment, at least one processor 3752 may be packaged and configured together with compute logic 3782 to practice aspects of various example embodiments described herein to form a system-in-package (SIP) or a system-on-chip (SoC).

[0073] The system 3750 includes processor circuitry in the form of one or more processors 3752. The processor circuitry 3752 includes circuitry such as, but not limited to, one or more processor cores and one or more of cache memory, low-dropout voltage regulators (LDOs), interrupt controllers, serial interfaces such as SPI, I2C, or a general-purpose programmable serial interface circuit, real-time clock (RTC), timer counters including interval and watchdog timers, general-purpose I / O, memory card controllers such as Secure Digital / Multimedia Card (SD / MMC) or the like, interfaces, Mobile Industry Processor Interface (MIPI) interfaces, and Joint Test Access Group (JTAG) test access ports.In some implementations, processor circuitry 3752 may include one or more hardware accelerators (e.g., the same as or similar to acceleration circuitry 3764), which may be microprocessors, programmable processing devices (e.g., FPGA, ASIC, etc.), or the like. The one or more accelerators may include, for example, computer vision and / or deep learning accelerators. In some implementations, processor circuitry 3752 may include on-chip memory circuitry, which may include any suitable volatile and / or non-volatile memory, such as DRAM, SRAM, EPROM, EEPROM, flash memory, solid-state memory, and / or any other type of memory device technology, such as those discussed herein.

[0074] The processor circuitry 3752 may include, for example, one or more processor cores (CPUs), application processors, GPUs, RISC processors, Acorn RISC Machine (ARM) processors, CISC processors, one or more DSPs, one or more FPGAs, one or more PLDs, one or more ASICs, one or more baseband processors, one or more radio frequency integrated circuits (RFICs), one or more microprocessors or controllers, a multi-core processor, a multi-threaded processor, an ultra-low voltage processor, an embedded processor, or any other known processing elements, or any suitable combination thereof.The processors (or cores) 3752 may be coupled to or include memory / storage and may be configured to execute instructions stored in the memory / storage to enable execution of various applications or operating systems on the platform 3750. The processors (or cores) 3752 are configured to execute application software to provide a specific service to a user of the platform 3750. In some embodiments, the one or more processors 3752 may be one or more special-purpose processors / controllers configured (or configurable) to operate according to the various embodiments herein.

[0075] As examples, the one or more processors 3752 may include an Intel® Architecture Core™-based processor, such as an i3, i5, i7, or i9-based processor; an Intel® microcontroller-based processor, such as a Quark™, ​​Atom™, or other MCU-based processor; one or more Pentium® processors, Xeon® processors, or other such processor available from Intel® Corporation, Santa Clara, California. However, any number of other processors may be used, such as one or more of an Advanced Micro Devices (AMD) Zen® architecture, such as one or more Ryzen® or EPYC® processors, accelerated processing units (APUs), MxGPUs, one or more Epyc® processors, or the like. one or more A5-A12 and / or S1-S4 processors from Apple® Inc., one or more Snapdragon™ or Centriq™ processors from Qualcomm® Technologies, Inc., one or more Texas Instruments, Inc.® Open Multimedia Applications Platform (OMAP)™ processors; a MIPS-based design from MIPS Technologies, Inc., such as MIPS Warrior M-Class, Warrior I-Class, and Warrior P-Class processors; an ARM-based design licensed from ARM Holdings, Ltd., such as the ARM Cortex-A, Cortex-R, and Cortex-M processor families; the ThunderX2® provided by Cavium™, Inc.; or the like.

[0076] In some implementations, the processor(s) 3752 and / or other components of the system 3750 may be part of a system-on-chip (SoC), a system-in-package (SIP), a multi-chip package (MCP), and / or the like, where the processor(s) 3752 and other components are formed into a single integrated circuit or a single package, such as the Edison™ or Galileo™ SoC boards from Intel® Corporation. Other examples of the processor(s) 3752 are mentioned elsewhere in this disclosure. In embodiments, two or more components of the system 3750 may be located on different dies coupled to the same base die. The base die may include resonant rings of an ROA, as described herein. The dies may tap the clock signal from the resonant rings at deterministic phase points, e.g., for D2D I / O communication and / or other purposes.

[0077] The system 3750 may include or be coupled to acceleration circuitry 3764, which may be implemented by one or more artificial intelligence (AI) / machine learning (ML) accelerators, a neural network, neuromorphic hardware, an FPGA, an array of GPUs, one or more SoCs (including programmable SoCs), one or more CPUs, one or more digital signal processors, dedicated ASICs (including programmable ASICs), PLDs such as complex PLDs (CPLDs) or highly complex PLDs (HCPLDs), and / or other forms of specialized processors or circuitry configured to perform one or more specialized tasks. These tasks may include AI / ML processing (e.g., including training, inference, and classification operations), visual computing, network computing, object detection, rule analysis, or the like.In FPGA-based implementations, acceleration circuitry 3764 may include logic blocks or a logic fabric and other interconnected resources that can be programmed (configured) to perform various functions, such as the procedures, methods, functions, etc. of the various embodiments discussed herein. In such implementations, acceleration circuitry 3764 may also include memory cells (e.g., EPROM, EEPROM, flash memory, static memory (e.g., SRAM, anti-fuses, etc.) used to store logic blocks, logic fabric, data, etc. in LUTs, and the like.

[0078] In some implementations, processor circuitry 3752 and / or acceleration circuitry 3764 may include hardware elements specifically tailored for machine learning and / or artificial intelligence (AI) functionality. In these implementations, processor circuitry 3752 and / or acceleration circuitry 3764 may be or include an AI engine chip capable of executing many different types of AI instruction sets once loaded with the appropriate weights and training code. Additionally or alternatively, processor circuitry 3752 and / or acceleration circuitry 3764 may be or include AI accelerators, which may be one or more of the aforementioned hardware accelerators configured for hardware acceleration of AI applications.As examples, this one or more processors or accelerators may be a cluster of artificial intelligence (AI) GPUs, tensor processing units (TPUs) developed by Google® Inc., Real AI Processors (RAPs™) provided by AlphaICs®, Nervana™ neural network processors (NNPs) provided by Intel® Corp., Intel® Movidius™ Myriade™ X vision processing unit (VPU), NVIDIA® PX™-based GPUs, the NM500 chip provided by General Vision®, Hardware 3 provided by Tesla®, Inc., an Epiphany™-based processor provided by Adapteva®, or the like.In some embodiments, the processor circuitry 3752 and / or the acceleration circuitry 3764 and / or the hardware accelerator circuitry may be implemented as AI accelerating coprocessor(s), such as the Hexagon 685 DSP provided by Qualcomm®, the PowerVR 2 Ion neural network accelerator (NNA) provided by Imagination Technologies Limited®, the neural engine core within the Apple® A11 or A12 Bionic SoC, the neural processing unit (NPU) within the HiSilicon Kirin 3770 provided by Huawei®, and / or the like. In some hardware-based implementations, individual subsystems of the System 3750 may be operated by the respective AI acceleration coprocessor(s), AI GPU(s), TPU(s), or hardware accelerators (e.g., FPGAs, ASICs, DSPs, SoCs, etc.), etc., connected to appropriate logic blocks, one or more bitstreams, etc.configured to perform their respective functions.

[0079] The system 3750 also includes a system memory 3754. Any number of memory devices may be used to provide a given amount of system memory. As examples, the memory 3754 may be or include volatile memory, such as random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), RAMBUS® dynamic random access memory (RDRAM®), and / or any other desired type of volatile memory device. Additionally or alternatively, the memory 3754 may be or include non-volatile memory, such as read-only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable memory (EEPROM), flash memory, non-volatile RAM, ferroelectric RAM, phase-change memory (PCM), flash memory, and / or any other desired type of non-volatile memory device.Access to the 3754 memory is controlled by a memory controller. The individual memory devices can be of any number of different package types, such as single die package (SDP), dual die package (DDP), or quad die package (Q17P). Any number of other memory implementations can be used, such as dual inline memory modules (DIMMs) of various varieties, including microDIMMs or miniDIMMs, among others.

[0080] The storage circuitry 3758 provides persistent storage of information such as data, applications, operating systems, and so on. In one example, the storage 3758 may be implemented via a solid-state drive (SSDD) and / or high-speed electrically erasable memory (commonly referred to as "flash memory"). Other devices that may be used for the storage 3758 include flash memory cards such as SD cards, microSD cards, XD image cards, and the like, and USB flash drives. In one example, the storage device may be or include storage devices that use chalcogenide glass, multi-threshold NAND flash memory, NOR flash memory, single- or multi-level phase-change memory (PCM), resistive memory, nanowire memory, ferroelectric transistor random access memory (FeTRAM), antiferroelectric memory, magnetoresistive random access memory (MRAM),Memory incorporating memristor technology, phase-change RAM (PRAM), resistive memory, including metal-oxide-based, oxygen-vacancy-based, and conductive bridge random access memory (CB-RAM), or spin-transfer torque MRAM (STT-MRAM), a spintronic magnetic junction memory-based device, a magnetic tunnel junction (MTJ)-based device, a domain wall (DW) and spin-orbit transfer (SOT)-based device, a thyristor-based memory device, a hard disk drive (HDD), a micro-HDD, or a combination thereof, and / or any other memory. Memory circuitry 3754 and / or storage circuitry 3758 may also include three-dimensional (3D) crosspoint memory (XPOINT memory) from Intel® and Micron®.

[0081] Memory circuitry 3754 and / or storage circuitry 3758 are configured to store computational logic 3783 in the form of software, firmware, microcode, or hardware instructions to implement the techniques described herein. Computational logic 3783 may be employed to store working copies and / or permanent copies of programming instructions or data for generating the programming instructions for operating various components of system 3700 (e.g., drivers, libraries, application programming interfaces (APIs), etc.), an operating system of system 3700, one or more applications, and / or for executing the embodiments discussed herein.Computation logic 3783 may be stored or loaded into memory circuitry 3754 as instructions 3782 or data for generating instructions 3782, which are then accessed for execution by processor circuitry 3752 to perform the functions described herein. Processor circuitry 3752 and / or acceleration circuitry 3764 access memory circuitry 3754 and / or storage circuitry 3758 via interconnect (IX) 3756. Instructions 3782 instruct processor circuitry 3752 to perform a specific sequence or flow of actions, such as described with reference to one or more flowcharts and one or more block diagrams of operations and functionality presented above.The various elements may be implemented by assembly instructions supported by processor circuitry 3752, or high-level languages ​​that can be compiled into instructions 3781, or data for generating instructions 3781 to be executed by processor circuitry 3752. The permanent copy of the programming instructions may be placed in persistent storage devices of storage circuitry 3758 at the factory or on-site, for example, through a distribution medium (not shown), through a communications interface (e.g., from a distribution server (not shown)), over-the-air (OTA), or any combination thereof.

[0082] The IX 3756 couples the processor 3752 to communication circuitry 3766 for communications with other devices, such as a remote server (not shown), and the like. The communication circuitry 3766 is a hardware element or collection of hardware elements used to communicate across one or more networks 3763 and / or with other devices. In one example, the communication circuitry 3766 is or includes transceiver circuitry configured to enable wireless communications using any number of frequencies and protocols, such as, for example, the Institute of Electrical and Electronics Engineers (IEEE) 802.11 (and / or variants thereof), IEEE 802.7.4, Bluetooth® and / or Bluetooth® Low Energy (BLE), ZigBee®, LoRaWAN™ (Long Range Wide Area Network), a cellular protocol such as 3GPP LTE and / or Fifth Generation (5G) / New Radio (NR), and / or the like. Additionally or alternatively, the communication circuitry 3766 is or includes one or more network interface controllers (NICs) to enable wired communication using, for example, an Ethernet connection, Controller Area Network (CAN), Local Interconnect Network (LIN), DeviceNet, ControlNet, Data Highway+, or PROFINET, among many others.

[0083] The IX 3756 also couples the processor 3752 to interface circuitry 3770, which is used to connect the system 3750 to one or more external devices 3772. The external devices 3772 may include, for example, sensors, actuators, positioning circuitry (e.g., Global Navigation Satellite System (GNSS) / Global Positioning System (GPS) circuitry), client devices, servers, network devices (e.g., switches, hubs, routers, etc.), integrated photonics devices (e.g., optical neural network (ONN) integrated circuit (IC) and / or the like), and / or other similar devices.

[0084] In some optional examples, various input / output (I / O) devices, referred to as input circuitry 3786 and output circuitry 3784 in Fig.37, may be present within or connected to the system 3750. The input circuitry 3786 and the output circuitry 3784 include one or more user interfaces that enable the user to interact with the platform 3750 and / or interfaces for peripheral components that enable the peripheral components to interact with the platform 3750. The input circuitry 3786 may include any physical or virtual means for accepting input, including, but not limited to, one or more physical or virtual buttons (e.g., a reset button), a physical keyboard, a keypad, a mouse, a touchpad, a touchscreen, microphones, a scanner, a headset, and / or the like. The output circuitry 3784 may be included to display or otherwise convey information, e.g.,Sensor readings, actuator position(s), or other similar information. Data and / or graphics may be displayed on one or more user interface components of output circuitry 3784. Output circuitry 3784 may include any number and / or combinations of audible or visual displays, including, but not limited to, one or more simple visual outputs / indicators (e.g., binary status indicators (e.g., light-emitting diodes (LEDs)) and multi-character visual outputs, or more complex outputs, such as display devices or touch screens (e.g., liquid crystal displays (LCDs), LED displays, quantum dot displays, projectors, etc.), where the output of characters, graphics, multimedia objects, and the like is generated or produced from the operation of platform 3750.The output circuitry 3784 may also include speakers and / or other audio emission devices, one or more printers, and / or the like. Additionally or alternatively, one or more sensors may be used as the input circuitry 3784 (e.g., an image capture device, motion capture device, or the like), and one or more actuators may be used as the output device circuitry 3784 (e.g., an actuator for providing haptic feedback or the like). Peripheral component interfaces may include, but are not limited to, a non-volatile memory port, a USB port, an audio jack, a power supply interface, etc.In some embodiments, display or console hardware may be used in the context of the present system to provide output and receive input of an edge computing system; manage components or services of an edge computing system; identify a state of an edge computing component or service; or perform any other number of management or administration functions or service use cases.

[0085] The components of the System 3750 can communicate through the IX 3756. The IX 3756 can incorporate any number of technologies, including ISA, enhanced ISA, I2C, SPI, point-to-point interfaces, power management bus (PMBus), PCI, PCIe, PCIx, Intel® UPI, Intel® Accelerator Link, Intel® CXL, CAPI, OpenCAPI, Intel® QPI, UPI, Intel® OPA IX, RapidIO™ System IXs, CCIX, Gen Z Consortium IXs, a HyperTransport interconnect, NVLink provided by NVIDIA®, a Time-Triggered Protocol (TTP) system, a FlexRay system, PROFIBUS, and / or any number of other IX technologies. The IX 3756 can be a proprietary bus, used, for example, in an SoC-based system.

[0086] The number, capability, and / or capacity of the elements of system 3700 may vary depending on whether computing system 3700 is used as a stationary computing device (e.g., a server computer in a data center, a workstation, a desktop computer, etc.) or a mobile computing device (e.g., a smartphone, tablet computing device, laptop computer, gaming console, IoT device, etc.). In various implementations, computing device system 3700 may include one or more components of a data center, a desktop computer, a workstation, a laptop, a smartphone, a tablet, a digital camera, a smart device, a smart home hub, a network device, and / or any other device / system that processes data. Examples

[0087] Some non-limiting examples of various embodiments are provided below.

[0088] Example 1 is a device comprising: a base die including resonant rings of respective rotary oscillators, wherein the resonant rings of different rotary oscillators are shorted together to form a rotary oscillator array (ROA); and a first die and a second die coupled to the base die, wherein the first die is to tap a clock signal from the ROA and transmit the serialized data based on the tapped clock signal to the second die.

[0089] Example 2 may include the device of Example 1 or any other example herein, wherein the resonant rings of the respective rotary oscillators include a first ring and a second ring coupled together, the rotary oscillators further including one or more pairs of cross-coupled inverters coupled between the first ring and the second ring.

[0090] Example 3 may include the device of Example 2 or any other example herein, wherein the inverters are included in the base die.

[0091] Example 4 may include the device of Example 2 or any other example herein, wherein the inverters are included in at least one of the first die or the second die.

[0092] Example 5 may include the device of Example 4 or any other example herein, wherein the inverters are coupled to the resonant rings via microbumps.

[0093] Example 6 may include the device of Examples 2-5 or any other example herein, wherein the rotary oscillators include a first rotary oscillator and a second rotary oscillator, the first ring of the first rotary oscillator being shorted to the second ring of the second rotary oscillator and the second ring of the first rotary oscillator being shorted to the first ring of the second rotary oscillator.

[0094] Example 7 may include the device of Examples 1-6 or any other example herein, wherein the clock signal is a first clock signal, and wherein the second die is to tap a second clock signal from the ROA and receive the data based on the second clock signal.

[0095] Example 8 may include the device of Example 7 or any other example herein, wherein the first die includes transmit circuitry having one or more serializers to serialize the data based on the first clock signal for transmission to the second die, and wherein the second die includes receive circuitry having one or more deserializers to deserialize the data.

[0096] Example 9 may include the device of Example 7-8 or any other example herein, wherein the first clock signal has a same phase as the second clock signal.

[0097] Example 10 may include the device of Example 7-8 or any other example herein, wherein the first clock signal has a different phase than the second clock signal.

[0098] Example 11 may include the device of Example 10 or any other example herein, wherein the second clock signal is 45 to 135 degrees ahead in phase of the first clock signal.

[0099] Example 12 may include the apparatus of Example 7, 8, 10, or 11, wherein the data is transmitted over a communication bus having multiple channels utilizing respective pairs of tap points, the respective pairs of tap points having different phase differences between the first and second clock signals.

[0100] Example 13 may include the device of Example 1-12 or any other example herein, wherein the rotary oscillators are traveling wave rotary oscillators.

[0101] Example 14 may include the device of Example 1-9 or any other example herein, wherein the rotary oscillators are standing wave rotary oscillators.

[0102] Example 15 may include the device of Example 1-14 or any other example herein, wherein at least one of the resonant rings has an irregular shape.

[0103] Example 16 may include the device of Example 15 or any other example herein, wherein the irregular shape is a non-rectangular rectilinear shape.

[0104] Example 17 may include the device of Example 1-16 or any other example herein, wherein at least one of the resonant rings has a rectangular shape.

[0105] Example 18 may include the device of Examples 1-17 or any other example herein, wherein at least one of the rotary oscillators is operable in a traveling wave mode and a standing wave mode.

[0106] Example 19 may include the apparatus of Example 18 or any other example herein, wherein the rotary oscillators include one or more switches coupled between the first ring and the second ring of the respective rotary oscillators to control whether the respective rotary oscillators are in traveling wave mode or standing wave mode.

[0107] Example 20 may include the device of claim 19 or any other example herein, wherein the switches are to be open when the respective rotary oscillator is in traveling wave mode, and wherein a selected one of the switches is to be closed when the respective rotary oscillator is in standing wave mode.

[0108] Example 21 may include the device of claim 1-20 or any other example herein, wherein a first resonant ring of the resonant rings has a first long side below the first die and a second long side below the second die.

[0109] Example 22 may include the device of claim 21, wherein the first resonant ring is rectangular and further includes a first short side coupled between the first and second long sides and a second short side coupled between the first and second long sides.

[0110] Example 23 may include the device of claim 21-22 or any other example herein, wherein the first long side is at least partially located below a first D2D PHY circuitry of the first die, and wherein the second long side is at least partially located below a second D2D PHY circuitry of the second die.

[0111] Example 24 may include the device of claim 1-23 or any other example herein, wherein the rotary oscillators are standing wave oscillators, and wherein the rotary oscillators include one or more clock recovery circuits coupled to the resonant rings, wherein a first clock recovery circuit of the one or more clock recovery circuits is to generate the clock signal.

[0112] Example 25 may include the device of Example 24 or any other example herein, wherein the first clock recovery circuit is coupled to the first and second rings of the respective resonant rings.

[0113] Example 26 may include the device of Example 25, wherein the clock recovery circuits are to generate a square wave.

[0114] Example 27 may include a multi-die system comprising: a base die including resonant rings of respective rotary oscillators, wherein the resonant rings of different rotary oscillators are shorted together to form a rotary oscillator array (ROA); a first die including transmit circuitry to: retrieve a first clock signal from the ROA, serialize data based on the first clock signal, and transmit the serialized data to the second die via a communication bus; and a second die including receive circuitry to receive the serialized data via the communication bus, retrieve a second clock signal from the ROA, and deserialize the data based on the second clock signal.

[0115] Example 28 may include a device comprising: a base die including a resonant ring structure of a rotary oscillator; a first die coupled to the base die, the first die including transmit circuitry above a resonant ring, the transmit circuitry to tap a first clock signal from the resonant ring and transmit the data based on the first clock signal; and a second die coupled to the base die, the second die including receive circuitry above the resonant ring, and the second die including receive circuitry to tap a second clock signal from the resonant ring and receive the data based on the second clock signal.

[0116] Example 29 may include the apparatus of Example 28, wherein the transmit circuitry is located above a first long edge of the resonant ring structure and is to tap the first clock signal from the first long edge, and wherein the receive circuitry is located above a second long edge of the resonant ring structure and is to tap the second clock signal from the second long edge.

[0117] Example 30 may include the device of Example 28 or any other example herein, wherein the rotary oscillator is a traveling wave rotary oscillator.

[0118] Example 31 may include the device of Example 30 or any other example herein, wherein the first clock signal has a different phase than the second clock signal.

[0119] Example 32 may include the apparatus of Example 31 or any other example herein, wherein the second clock signal is 45 to 135 degrees ahead in phase of the first clock signal.

[0120] Example 33 may include the device of Example 28 or any other example herein, wherein the rotary oscillator is a standing wave rotary oscillator.

[0121] Example 34 may include the apparatus of Example 33 or any other example herein, wherein the rotary oscillator further includes a clock recovery circuit coupled to a first ring and a second ring of the resonant ring structure to generate a square wave signal as a clock signal.

[0122] Example 35 may include implementing any of Examples 28-34 or any other example herein, wherein the transmit circuitry includes one or more serializers to serialize the data based on the first clock signal, and wherein the receive circuitry includes one or more deserializers to deserialize the data based on the second clock signal.

[0123] Example 36 may include a computer system comprising: a multi-die system (MDS) and one or more antennas coupled to the MDS to enable the computer system to wirelessly communicate with another device.The MDS may include: a base die including a resonant ring structure of a traveling wave rotary oscillator (RTWO) array; a first die coupled to the base die, the first die including transmit circuitry, the transmit circuitry to sample a first clock signal from the resonant ring and serialize data based on the first clock signal; and a second die coupled to the base die, the second die including receive circuitry above the resonant ring, the receive circuitry to sample a second clock signal from the resonant ring and deserialize the data based on the second clock signal, the second clock signal having a different phase than the first clock signal.

[0124] Example 37 may include the system of Example 36, wherein the second clock signal is 45 to 135 degrees ahead of the first clock signal in phase.

[0125] Example 38 may include the system of example 36 or 37, wherein the data is transmitted over a communication bus having multiple channels using respective pairs of tap points, the respective pairs of tap points having different phase differences between the respective first and second clock signals.

[0126] Example 39 may include the system of examples 36 to 38, wherein the transmit circuitry and the receive circuitry are to tap the respective first and second clock signals from the same ring of the resonant ring structure.

[0127] Example 40 may include the system of examples 36 to 38, wherein the transmit circuitry and the receive circuitry are to tap the respective first and second clock signals from different rings of the resonant ring structure.

[0128] Example 41 may include a computer system comprising: the device of any of Examples 1-40; and at least one of a memory, a communications interface, a radio frequency circuit, or one or more antennas coupled to the multi-die system.

[0129] Although specific embodiments have been illustrated and described herein for descriptive purposes, this application is intended to cover any adaptations or variations of the embodiments discussed herein. Accordingly, it is expressly intended that the embodiments described herein be limited only by the claims.

[0130] When the disclosure recites "a" or "a first" element, or the equivalent thereof, such disclosure includes one or more such elements, and two or more such elements are neither required nor precluded. Furthermore, ordinal designations (e.g., first, second, or third) for identified elements are used to distinguish among the elements and do not indicate or imply a required or limited number of such elements, nor do they indicate a particular position or order of such elements, unless expressly stated otherwise.

Claims

[1] Facility comprising: a base die including resonant rings of respective rotary oscillators, wherein the resonant rings of different rotary oscillators are shorted together to form a rotary oscillator array (ROA); and a first die and a second die coupled to the base die, wherein the first die is to tap a first clock signal from the ROA and transfer the data to the second die based on the first clock signal; and wherein the second die is to take a second clock signal from the ROA and receive the data based on the second clock signal. [2] The device of claim 1, wherein the resonant rings of the respective rotary oscillators include a first ring and a second ring cross-coupled to each other, the rotary oscillators further including one or more pairs of cross-coupled inverters coupled between the first ring and the second ring. [3] The device of claim 1, wherein at least one of the resonant rings has a non-rectangular rectilinear shape. [4] The device of claim 1, wherein at least one of the resonant rings has a rectangular shape. [5] The device of claim 1, wherein a first resonant ring of the resonant rings has a first long side below the first die and a second long side below the second die. [6] The device of claim 5, wherein the first resonant ring is rectangular and further includes a first short side coupled between the first and second long sides and a second short side coupled between the first and second long sides. [7] The device of claim 5, wherein the first long side is at least partially located beneath a first D2D PHY circuitry of the first die, and wherein the second long side is at least partially located beneath a second D2D PHY circuitry of the second die. [8] The device of claim 1, wherein the first clock signal has a same phase as the second clock signal. [9] Device according to one of claims 1-8, wherein the rotary oscillators are traveling wave rotary oscillators. [10] The device of claim 9, wherein the first clock signal has a different phase than the second clock signal. [11] The device of claim 10, wherein the second clock signal is 45 to 135 degrees ahead of the first clock signal in phase. [12] The device of claim 11, wherein the data is transmitted over a communication bus having multiple channels using respective pairs of tap points, the respective pairs of tap points having different phase differences between the first and second clock signals. [13] Device according to one of claims 1-8, wherein the rotary oscillators are standing wave rotary oscillators. [14] The device of claim 13, wherein the rotary oscillators include one or more clock recovery circuits coupled to the resonant rings, wherein a first clock recovery circuit of the one or more clock recovery circuits is coupled to the first and second rings of the respective resonant rings to generate a square wave signal as the clock signal. [15] The device of claim 1, wherein at least one of the rotary oscillators is operable in a traveling wave mode and a standing wave mode, and wherein the rotary oscillators include one or more switches coupled between the first ring and the second ring of the respective rotary oscillators, and wherein the device further comprises control circuitry to control the switches to be open when the respective rotary oscillator is in the traveling wave mode and to control the switches to close a selected one of the switches when the respective rotary oscillator is in the standing wave mode. [16] The device of claim 1, wherein the first die includes one or more serializers for serializing the data for transmission based on the first clock signal, and wherein the second die includes one or more deserializers for deserializing the data. [17] Facility comprising: a base die including a resonant ring structure of a rotary oscillator; a first die coupled to the base die, the first die including transmit circuitry above a resonant ring, the transmit circuitry to tap a first clock signal from the resonant ring and transmit the data over a communication bus based on the first clock signal; a second die coupled to the base die, the second die including receive circuitry above the resonant ring and coupled to the communication bus, the second die to tap a second clock signal from the resonant ring and receive the data based on the second clock signal. [18] The device of claim 17, wherein the transmit circuitry is located above a first long edge of the resonant ring structure and is to tap the first clock signal from the first long edge, and wherein the receive circuitry is located above a second long edge of the resonant ring structure and is to tap the second clock signal from the second long edge. [19] The device of claim 17, wherein the rotary oscillator is a traveling wave rotary oscillator. [20] The device of claim 19, wherein the first clock signal has a different phase than the second clock signal. [21] The device of claim 20, wherein the second clock signal is 45 to 135 degrees ahead of the first clock signal in phase. [22] The device of claim 17, wherein the rotary oscillator is a standing wave rotary oscillator. [23] The device of claim 22, wherein the rotary oscillator further includes a clock recovery circuit coupled to a first ring and a second ring of the resonant ring structure for generating a square wave signal as the clock signal. [24] The device of any of claims 17-23, wherein the transmit circuitry includes one or more serializers for serializing the data based on the first clock signal for transmission over the communication bus, and wherein the receive circuitry includes one or more deserializers for deserializing the data based on the second clock signal. [25] Computer system comprising: a Multi-Die System (MDS) that includes: a base die including a resonant ring structure of a traveling wave rotary oscillator (RTWO) array; a first die coupled to the base die, the first die including transmit circuitry, and the transmit circuitry to tap a first clock signal from the resonant ring structure and serialize data based on the first clock signal; and a second die coupled to the base die, the second die including receive circuitry, the receive circuitry to tap a second clock signal from the resonant ring structure and deserialize the data based on the second clock signal, and the second clock signal having a different phase than the first clock signal; and one or more antennas coupled to the MDS to enable the computer system to communicate wirelessly with another device. [26] The system of claim 25, wherein the second clock signal is 45 to 135 degrees ahead of the first clock signal in phase. [27] A system according to claim 25 or 26, wherein the data is transmitted over a communication bus having a plurality of channels using respective pairs of tap points, the respective pairs of tap points having different phase differences between the respective first and second clock signals. [28] A system according to any one of claims 25-27, wherein the transmit circuitry and the receive circuitry are to tap the respective first and second clock signals from the same ring of the resonant ring structure. [29] A system according to any one of claims 25-27, wherein the transmit circuitry and the receive circuitry are to tap the respective first and second clock signals from different rings of the resonant ring structure.

Images