Electro-photonic networks for machine learning
Through the hybrid electron-photon network and efficient photon channel connection, the problems of high data transmission energy and inefficient multiplication-accumulation operations in artificial intelligence computing systems are solved, and efficient calculation of AI models is realized.
Patent Information
- Application Number
- CN202280020819.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2021-06-18
- Filing Date
- 2022-06-17
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2042-06-17
AI Technical Summary
In the prior art, data transmission between chips in artificial intelligence computing systems consumes a lot of energy, and multiplication-accumulation operations are power-efficient in hardware implementation, making it difficult to meet the rapidly growing complexity needs of AI models.
Using a hybrid electron-photon network, the processing elements are connected through bidirectional photon channels, and the photon channels are used for data transmission. Combined with an efficient dot product engine and clock timing scheme, it reduces the power consumption of data movement and improves processing speed.
The data movement power and processing speed are improved in the AI model are achieved, which significantly reduces the energy consumption of multiplication-accumulation operations and improves the power efficiency of the system.
Smart Images

Figure CN117063185B_ABST
Abstract
Description
[0001] Related Applications
[0002] This application claims the benefit of priority to U.S. Provisional Patent Application Serial No. 63 / 212,353, filed on June 18, 2021, which is incorporated herein by reference in its entirety. Background Art
[0003] The demand for artificial intelligence (AI) computing, such as machine learning (ML) and deep learning (DL), is increasing faster than the available processing power can meet. This rising demand and the growing complexity of AI models have prompted the need to connect many chips into a system where the chips can send data between each other with low latency and high speed. In currently known methods, a serializer / deserializer (SerDes) block is used to connect chips, which converts parallel messages into a serial bit stream that can be sent through electrical interconnects or optical fibers between chips. In such systems, a distinction is made between on-chip and off-chip communications. The computing elements on the chip transmit packets via metal interconnects, while messages (e.g., packets) to another chip are moved to the site of the SerDes interface through chip-level interconnects, where the data is converted into a bit stream and transmitted. In the receiving direction, the bits arrive at the optical fiber or electrical interconnect, are assembled into words, and are then transmitted to the destination processor or memory through the metal interconnects inside the chip. A lot of energy is consumed in both moving the data within the chip to the SerDes and then moving it from the SerDes to other chips in the system. Furthermore, currently known hardware implementations of ML models are relatively power-inefficient in performing multiply-accumulate (MAC) operations that are extensively used during ML model execution. BRIEF DESCRIPTION OF THE DRAWINGS
[0004] The various figures in the drawings merely illustrate various embodiments of the presently disclosed subject matter and should not be considered to limit the scope thereof.
[0005] Figure 1A is a diagram schematically illustrating components of an example system-in-package (SIP) in accordance with some embodiments.
[0006] Figure 1B is a schematic diagram showing a method according to some embodiments Figure 1A Illustration of an example electro-optical network within a SIP.
[0007] Figure 1C is a side view showing a method according to some embodiments Figure 1A and Figure 1B An illustration of an example structural implementation of a SIP.
[0008] Figure 2Ais a diagram showing a method according to some embodiments Figure 1A and Figure 1B An illustration of example electronic processing elements of a SIP, the elements including a message router.
[0009] Figure 2B is a diagram showing an example configuration of message routers and photonic channels in an electro-photonic network according to some embodiments.
[0010] Figure 3 is a diagram showing an example tensor engine according to some embodiments, which may be used in Figure 2A The electronic processing components shown are used within.
[0011] Figure 4 is a diagram showing an example digital neural network (DNN) according to some embodiments, which DNN can be used in Figure 2A The electronic processing components shown are used within.
[0012] Figure 5A is a diagram showing an example implementation of photonic components and electronic components of an electro-photonic network in side view, according to some embodiments.
[0013] Figure 5B FIG. 1 is a top view showing a method according to some embodiments of the present invention. Figure 5A Illustration of a photonic integrated circuit (PIC) of an example electro-photonic network.
[0014] Fig. 6A is a diagram showing an example group of photonic links connecting two processing elements within a SIP in accordance with some embodiments.
[0015] Figure 6B is a diagram showing an example system of two SIPs using wavelength division multiplexing (WDM) for inter-chip optical communication in accordance with some embodiments.
[0016] Figure 7 is a diagram showing an example electro-photonic network having a quadrilateral mesh topology in accordance with some embodiments.
[0017] Figure 8 is a diagram showing an example electro-photonic network having a wrapped torus mesh topology in accordance with some embodiments.
[0018] Fig. 9 is a diagram showing an example electro-photonic network with a scalable wrapped torus mesh topology in accordance with some embodiments.
[0019] Fig.10 is a diagram showing a conventional technique for clock distribution within a processor.
[0020] Fig.11 is a diagram showing an example of using a photonic structure to carry a root clock signal in accordance with some embodiments.
[0021] Fig.12 is a diagram showing an example architecture of a DNN according to some embodiments.
[0022] Fig.13 is a flowchart illustrating an example method for operating a DNN according to some embodiments.
[0023] Fig.14 is a flow chart illustrating an example method for clock signal distribution using an electro-optical network in accordance with some embodiments.
[0024] Fig.15 is a flow chart illustrating an example method for communicating messages within an electro-optical network in accordance with some embodiments.
[0025] Fig.16 is a flow chart illustrating an example method of implementing an artificial neural network (ANN) on an ML processor according to some embodiments. DETAILED DESCRIPTION
[0026] The following description contains specific information related to the embodiments in the present disclosure. The drawings in this application and the detailed descriptions attached thereto are for exemplary embodiments only. Unless otherwise specified, similar or corresponding elements in the figures may be indicated by similar or corresponding reference numerals. In addition, the drawings and illustrations in this application are generally not drawn to scale and are not intended to correspond to actual relative sizes.
[0027] The present disclosure provides a computing system implemented by one or more circuit packages (e.g., SIPs) that achieves reduced power consumption and / or increased processing speed. According to various embodiments, the power consumed, in particular, by data movement is reduced by maximizing data locality in each circuit package and reducing energy losses when data movement is required. Power-efficient data movement can then be achieved by moving data over small distances in the electronic domain while utilizing photon channels to move data where resistance in the electronic domain and / or the speed at which data can move in the electronic domain results in bandwidth limitations that cannot be overcome using existing electronic technology. Therefore, in some embodiments, each circuit package includes an electronic integrated circuit (EIC) that includes multiple circuit blocks (hereinafter referred to as "processing elements") that are connected to a hybrid electronic-photonic (or electro-photonic) network-on-chip (NoC) via bidirectional photon channels (e.g., implemented in a separate layer of the package or in a PIC in the chip). Multiple such NoCs can be connected into a larger electro-photonic network via inter-chip bidirectional photonic channels (e.g., implemented via optical fibers) between respective circuit packages to scale computing systems to arbitrary sizes without incurring significant power penalties. Further disclosed embodiments provide a novel circuit design for power-efficient performance of MAC operations (also referred to herein as a "dot product engine"), and a novel clock timing scheme that results in additional power savings.
[0028] Although the described computing systems and their various novel aspects are generally applicable to a wide range of processing tasks, they are particularly suitable for implementing ML models, especially ANNs. As applied to ANNs, the circuit packages and interconnect circuit package systems described herein are also referred to as "ML processors" and "ML accelerators", respectively. A neural network typically includes one or more layers of artificial neurons that calculate neuron output activations from the weighted sum of input activation sets (corresponding to MAC operations). For a given neural network, the activation flows between nodes and layers are fixed. In addition, once the training of the neural network is completed, the neuron weights in the weighted summation and any other parameters associated with the calculation of activations are also fixed. Therefore, the NoC described herein enables itself to implement a neural network by assigning neural nodes to processing elements, preloading the fixed weights associated with the nodes into the memory of the corresponding processing elements, and configuring the data routing between the processing elements based on a predetermined activation flow. The weighted summation can be efficiently performed using the disclosed dot product engine, which is also referred to as a "digital neural network (DNN)" herein because it is applicable to ANNs.
[0029] The foregoing high-level overview of various beneficial aspects and features of the disclosed computing system and underlying concepts will become more apparent from the following description of example embodiments.
[0030] Figure 1A 1 is a diagram schematically illustrating components of an example circuit package SIP 100 according to some embodiments. SIP 100 may be used, for example, as an ML processor. SIP 100 includes EIC 101, such as a digital and mixed signal application specific integrated circuit (ASIC), and PIC 102. EIC 101 and PIC 102 are formed in different layers of the SIP (respectively referred to herein as the "electronic circuit layer" and the "photonic circuit layer"), one layer stacked on top of the other, as further referenced below. Figure 1C As shown. One or more PICs 102 receive light from one or more laser sources, which may be integrated into the PIC 102 itself, or implemented separately from the PIC (inside or outside the SIP 100) and coupled to the PIC 102 via a suitable optical coupler. The optical coupler and the laser source are Figure 1A Omitted in, but in Figure 1B , Figure 1C and FIG. 5A to FIG. 6B Shown in.
[0031] EIC 101 includes a plurality of processing elements 104 that communicate with each other via bidirectional photonic channels implemented with optical waveguides in PIC 102. Processing elements 104 may (although they need not be in all embodiments) be electronic circuits that are identical (or at least substantially similar) in design, and, as shown, may form "tiles" of the same size arranged in a rectangular grid. (Hereinafter, the words "processing element" and "tile" are used synonymously). In the depicted example, EIC 101 has sixteen such processing elements or tiles 106, which are arranged in a four-by-four array, but the number and arrangement of tiles may generally vary. Neither the shape of the tiles nor the grid in which they are arranged is necessarily rectangular; for example, oblique quadrilateral, triangular, or hexagonal shapes and grids are also possible in principle. Furthermore, although tiling can make efficient use of available on-chip floor space, in all embodiments, processing elements 104 need not be equally sized and regularly arranged.
[0032] Each processing element 104 in EIC 101 includes one or more circuit blocks that function as processing engines. Figure 1A 2, each processing element 104 includes a dot product engine or DNN 106 and a tensor engine 108. The DNN 106 can perform fast MAC operations with reduced energy per MAC to perform convolution functions or dot product functions, for example, as conventionally used in neural networks. Figure 4 Details of an example DNN 106 are provided. The tensor engine 108 can be used to perform other non-MAC operations, for example, to implement nonlinear activation functions applied to weighted sums in a neural network. Figure 3 An example tensor engine 108 is described.
[0033] like Figure 1A As further shown in FIG. 2 , each processing element 104 includes a message router 110. The message router 110 interfaces with the photonic channels in the PIC 102 to facilitate data flow between the processing elements 104. In addition, the processing elements 104 may each have a memory system, for example, including a level 1 static random access memory (L1 SRAM) 112 and a level 2 static random access memory (L2 SRAM) 114. The L1 SRAM 112 is optional and, if included, may be used as scratch memory for each processing element 104. The L2 SRAM 114 may be used as the main memory for each processing element 104 and may store certain fixed operands used by the DNN 106 and the tensor engine 108, such as weights of a machine learning model that is physically very close to the DNN 106 and the tensor engine 108. The L2 SRAM 114 may also store any intermediate results used in executing the machine learning model or other calculations.
[0034] The EIC 101 may further include optional elements such as a peripheral component interconnect express (PCIE) interface 122, an advanced RISC machine (ARM) core 124, an image processor 126, and / or an external memory controller 128, which may support dynamic random access memory (DRAM), non-volatile random access memory (NVRAM), static random access memory (SRAM), or another type of memory. The PCIE interface 122 generally implements electrical interconnection between the EIC 101 and external components. For example, weights stored in the L2SRAM may be received from an external component (such as DRAM) via the PCIE interface 122. Similarly, the ARM core 124 may interface / interface a memory device external to the SIP 100, and may process image data or perform other computing tasks. The image processor 126 may process an image received from a memory device or another processor, wherein the image may originate from a camera. The memory controller 128 may communicate with a high bandwidth memory (HBM) that may be included in the SIP 100, and then interface with a DRAM external to the SIP 100. In some examples, EIC 101 includes multiple memory controllers for different types of memory.
[0035] Figure 1B is a schematic diagram showing a method according to some embodiments Figure 1A1 is a diagram of an example electro-optical network (or NoC) 130 within a SIP 100 of FIG. 1. Each processing element 104, and more specifically, a message router 110 in each processing element 104, functions as a node 131 in the network 130. Connections between these nodes 131 are established via bidirectional photonic channels, each channel interfacing with a message router of one processing element at one end and with a message router of another processing element at the other end. (The physical implementation of the photonic channels and the interface between the channels and the message routers will be referred to below.) Figure 5A and Figure 5B Message passing between any two processing elements 104 that do not share a bidirectional photonic channel directly connecting them can be accomplished by multiple "hops" along a path through one or more intervening nodes 131 within the electro-optical network 130; each such hop from one node to the next involves converting a message from the electronic domain into the photonic domain and back into the electronic domain. Thus, the network 130 is a hybrid electro-optical network.
[0036] exist Figure 1B , each pair of immediately adjacent processing elements (or tiles) 104 in a regular grid is connected by a photon channel, thereby forming a rectangular grid topology of the electro-optical network 130. However, other network topologies are also possible, for example, due to omitting connections between adjacent processing element pairs and / or increasing direct connections between non-adjacent processing elements. In addition, even if the processing elements 104 are physically arranged in two dimensions in a single EIC 101, the photon channels can be collectively configured to establish a topologically three-dimensional electro-optical network.
[0037] like Figure 1B As further shown in , in addition to interconnecting the processing elements 104, the photon channels in the PIC 102 can also connect the processing elements 104 (e.g., along the periphery of the tiled arrangement) to other elements of the EIC 101, such as to the PCIE interface 122, the ARM core 124, the image processor 126, and / or the memory controller 128. Alternatively, some electronic components of the EIC can be electrically interconnected. For example, the PCIE interface 122 can be electrically connected to one of the processing elements 104 within the electro-photonic NoC, which can be used as an entry point for communication with all other processing elements 104. According to various embodiments described herein, both electrical signal routing and photon signal routing are adopted. Signal routing tasks can be assigned between electrical (or electronic) paths and photon paths in a manner and for reasons that will be apparent from the following discussion.
[0038] PIC 102 may also include one or more optical coupling structures for making off-chip optical connections (e.g., via optical fibers). Fiber optic connections may be made by several means; example optical coupling structures for fiber optic connections include a fiber attachment unit (FAU) located above a grating coupler, or an edge coupler. Figure 1B Two FAUs 132 are shown; one FAU may be used to supply optical power to PIC 102 from an external laser source to drive photons in PIC 102, while the other FAU may facilitate chip-to-chip interconnect to another electro-photonic network in another SIP for both transmit and receive communications. Figure 1B Two FAUs 132 are shown in FIG. 1 , but other numbers of FAUs may be used in different examples.
[0039] Figure 1C is a side view showing a method according to some embodiments Figure 1A and Figure 1B 1 is a diagram of an example structural implementation of a SIP 100. In this example, an EIC (e.g., ASIC) 101 and a PIC 102 are formed in separate semiconductor chips (typically silicon chips, although other semiconductor materials are contemplated). The PIC 102 is disposed directly on a substrate 140, with solder bumps shown for subsequent mounting to a printed wiring board (PCB). The EIC 101 and a FAU 132 that connects the PIC 102 to an optical fiber 133 are disposed on top of the PIC 102 and optically connected to the PIC 102. Optionally, as shown, the SIP 100 may further include a memory chip 142 that is placed on top of the PIC 102 adjacent to the EIC 101.
[0040] As will be appreciated by one of ordinary skill in the art, the depicted structure of SIP 100 is only one of several possible ways to assemble and package the various components. In alternative embodiments, for example, EIC 101 may be provided on a substrate, with PIC 102 placed on top of EIC 101. In principle, as an alternative to implementing the electronic circuit layer and the photonic circuit layer as separate chips, it is also possible to create EIC 101 and PIC 102 in different layers of a single semiconductor chip. Furthermore, the photonic circuit layer may have multiple PICs in multiple sub-layers, for example, to reduce waveguide crossings. Furthermore, Figure 1C The structure depicted in can be modified to include multiple EICs 101 connected to a single PIC 102 and connected to each other via photonic channels in the PIC 102.
[0041] The EIC 101 and PIC 102 can be manufactured using standard wafer fabrication processes, including, for example, photolithographic patterning, etching, ion implantation, etc. In addition, in some embodiments, a heterogeneous material platform and integration process are used. For example, III-V semiconductor components can be used to implement various active photonic components, such as laser sources and / or optical modulators and photodetectors used in the photonic channel.
[0042] The laser source(s) may be implemented in the SIP 100, or externally. When implemented externally, the connection to the SIP 100 may be made optically, for example, using a grating coupler in the PIC below the FAU 132, such as Figure 1C As shown, or alternatively an edge coupler may be used. To implement a laser in the SIP 100, one option is to use an interposer containing several lasers that can be co-packaged with the PIC 102 and edge coupled. Alternatively, the lasers can be integrated directly into the PIC 102 using heterogeneous or homogeneous integration. Homogeneous integration allows the laser to be implemented directly in the silicon substrate in which the waveguides of the PIC 102 are formed, and allows lasers of different materials (such as, for example, indium phosphide (InP)) and architectures (such as quantum dot lasers). Heterogeneous assembly of lasers on the PIC 102 allows III-V semiconductors or other materials to be precisely attached to the PIC 102 and optically coupled to the waveguides implemented on the PIC 102.
[0043] Several SIPs 100, each including its own electro-photonic network 130, can be interconnected to create a single system that provides a larger electro-photonic network. For example, multiple SIPs configured as ML processors can be interconnected to form a larger ML accelerator. Photonic channels within several SIPs or ML processors, along with optical connections on PCBs, laser sources, passive optical components, and external optical fibers that can be utilized in various combinations and configurations with other photonic elements, form the photonic structure of a multi-SIP system or a multi-ML processor accelerator.
[0044] Figure 2A is a diagram illustrating an example electronic processing element 104 according to some embodiments, the electronic processing element 104 comprising Figure 1A and Figure 1B The message router 110 of the SIP 100. Figure 2A Further details are provided, particularly regarding message router 110 and its interface with the bidirectional photonic channels of PIC 102. Each bidirectional photonic channel comprises a unidirectional photonic link in each direction, each unidirectional link being implemented using an optical waveguide in PIC 102.
[0045] In one embodiment, a message containing packet data arrives via the photonic channel of the PIC 102 and is received at an optical-electrical (OE) interface between the PIC 102 and the router 110. The OE interface may be implemented, for example, by a photodetector (e.g., a photodiode) in the PIC 102 to convert the optical signal into an electronic signal in conjunction with associated electronic circuitry 200 in the router 110, which may include, for example, a transimpedance amplifier (TIA), an optional gain control for normalizing the signal level, and a slicer for extracting the bit stream. The message may then be electronically buffered in a register, such as a "first in, first out" (FIFO) register 202.
[0046] The electronic message router 110 includes a wiring system for examining the address contained in the message header (or, in an alternative embodiment, contained in the message payload) and determining which port and destination the message should be routed to. For example, if the destination is another processing element or tile within the electro-optical network, the message can be routed to the destination tile via an electro-optical (EO) interface between the router 110 and the PIC 102, where the message is converted back to the optical domain for transmission via another photonic channel. For example, an optical modulator within the PIC 102 can be used in conjunction with an associated driver wiring system (also referred to herein as a "modulator driver") 204 in the router to implement the EO interface; non-limiting examples of applicable modulator technologies include electro-absorption modulators (EAMs), Mach-Zehnder modulators, ring modulators, and quantum confined Stark effect electro-absorption modulators (QCCE EAMs). On the other hand, if the electronic message router 110 determines that the destination of the message is the tile memory (eg, L1 SRAM 112 or L2 SRAM 114 ), DNN 106 , or tensor engine 108 where the router 110 itself is located, the message is routed to the local port 206 .
[0047] The EIC side portions 200, 204 of the OE and EO interfaces between the message router 110 and a given bidirectional photon channel (in both directions) link, such as the TIA and other circuitry associated with the PIC side photodiode (collectively 200) and the driver circuitry associated with the PIC side optical modulator, are collectively referred to herein as the "photon channel interface" of the router 110. Although Figure 2A Only one photon channel interface is shown, but each router 110 typically includes multiple such interfaces to couple to multiple corresponding photon channels.
[0048] Figure 2B is a diagram illustrating an electro-optical network (such as, for example, Figure 1B1 and 2. In this example, the router 110 (with its corresponding photon channel 210) is a diagram of an example configuration of a message router 110 and a photon channel 210 in a network 130 shown in FIG. Figure 1B The processing elements or tiles 104 to which they belong are arranged in a two-dimensional, quadrilateral (more specifically, rectangular) array or grid, and in either dimension (e.g., Figure 2B The routers 110 of any two tiles that are adjacent to each other (horizontally or vertically) in the array are connected by bidirectional channels 210. Therefore, each router 110 in the interior of the array includes four photon channel interfaces 212 to four corresponding bidirectional photon channels 210, one for each of the four directions, which can be referred to as "north", "east", "south" and "west". Along the periphery of the array, the routers 110 only need to have two (at the corners) or three (at the edges between the corners) photon channel interfaces 212. However, as shown in the figure, as a result of using the same line design for all tiles 104, the routers 110 along the periphery can also be optionally equipped with four photon channel interfaces, which simplifies manufacturing. In addition, in some embodiments, the photon channel interfaces 214 along the periphery of the array can be used to establish chip-to-chip connections to other SIPs.
[0049] Electronic message router 110 can be used to route messages to destination addresses using various addressing schemes. Regardless of the addressing scheme, in contrast to the electrical interconnection on EIC 101, messages from tile to tile can be transmitted primarily or exclusively through an electro-optical network via a photon channel in PIC 102 (with optical to electrical and electrical to optical conversion at each router along the path). In one embodiment, a signed 5-bit packet data (e.g., extracted from a header, or alternatively, extracted from the payload of a message) provides a relative position (or distance) from the destination tile in the horizontal direction (east / west), while another signed 5-bit packet data provides a relative position (or distance) from the destination tile in the vertical direction (south / west). Of course, packet data of different sizes (number of bits) can also be used, depending on, for example, the number of tiles and the size of the address space generated thereby. As the message traverses routers 110 in different tiles, depending on which dimension the message is transmitted, for each jump, either the horizontal coordinate decreases or the vertical coordinate decreases. When both 5-bit packets providing the direction from the destination tile become zero, the message has reached the destination tile and is forwarded to the local port 206 of the router in that tile for processing. In some cases, messages are used to carry read and write memory transactions between tiles.
[0050] Although the paths that carry data between tiles are photonic paths, such as implemented by optical waveguides on PIC 102, the path setup packet data can be provided in electronic form for use by message router 110. In addition, even for the same tile message routing, the path setup packet data can be used to determine the path that can be a photonic path or an electrical path. Therefore, various embodiments adopt a hybrid approach in which both electronic components and photonic components are used in a hybrid electro-optical network-on-chip (EP-NoC) architecture to determine the destination, set up the delivery path, and deliver the message to the destination.
[0051] SIPs with electro-optical networks as described above lend themselves to efficient execution of ML models, such as neural networks. Neural networks include a plurality of nodes (also referred to herein as "neural nodes" to distinguish them from nodes of electro-optical networks), which are typically organized into layers, including an input layer, one or more hidden layers, and an output layer. In the hidden and output layers, each neural node is a computational unit that has as input a set of weights and an input activation from another node (e.g., a previous layer), and implements a transfer function that combines the weights and activations in a predetermined manner according to the ML model, thereby producing an output activation.
[0052] Data flows between nodes of the ML model in a deterministic manner. In addition, the weights associated with the nodes, which are adjustable parameters of the model, are typically determined during model training and remain fixed during inference computations when the trained model operates on inputs to generate outputs. When the ML model is executed on an ML processor or ML accelerator as described herein, these characteristics allow the ML model to be mapped onto an electro-optical network by fixedly assigning neural nodes to processing elements 104, and preloading the associated predetermined weights into the memory of the processing element or tile 104 to minimize data movement. Typically, when the ML model is distributed across multiple tiles 104 of an ML processor 100 or ML accelerator, each tile 104 may execute multiple neural nodes, one neural node, or a portion of a neural node that has been parallelized and distributed across several tiles 104. In some examples, each layer of the ML model is implemented by several tiles 104, wherein each tile 104 implements one or more neural nodes.
[0053] In some embodiments, during initialization of the inference operation of the ML processor 100 or ML accelerator, the weights are loaded into the memory (e.g., L2SRAM 114) of the assigned tile 104 only once, and the weights are not moved from tile to tile thereafter. The weights may be loaded from a device (e.g., memory) external to the ML processor 100 via the PCIE 122. In one embodiment, the PCIE 122 accesses the electro-optical network 130 of the ML processor through the tile 104 closest to the PCIE 122, and then photonically passes each weight to its corresponding destination tile 104 from tile to tile along a (e.g., predetermined) path through the electro-optical network. In other embodiments, during initialization of the ML processor 100, the weights are passed to each destination tile 104 using electronic connections on the EIC 101 that exist between tiles 104, rather than photonic channels located on the PIC 102. Regardless of whether the transfer of weights provided through the PCIE interface 122 is primarily accomplished photonically or electronically, the transfer and loading into each L2SRAM 114 is performed only once. In this way, significant power is saved because the weights remain static in the L2SRAM 114 of each tile 104.
[0054] Once the ML model has been preloaded into the ML processor by storing the weights associated with the neural nodes in the L2SRAM 114 of the corresponding tiles 104 assigned to the neural nodes, the ML model can be executed by streaming activations between the neural nodes and performing calculations on the streamed activations using the weights stored in the L2SRAM 114. The input activations are streamed to all tiles 104 assigned to the neural nodes in the ML model. The processing engines (e.g., DNN 106 and tensor engine 108) within these tiles 104 operate on the input activations and the preloaded weights to calculate the output activations of the neural nodes. For example, the DNN 106 can perform a MAC operation, and the tensor engine 108 can implement a non-linear activation function that operates on the result of the MAC operation, but other distributions of calculations between one or more processing engines in each tile 104 are also possible. The output activations generated for a given neural node in the tile 104 are then sent to the tile(s) 104 implementing the next(s) nodes in the ML model (e.g., within the next layer of the neural network) as their input activations. Activations flowing between neural nodes implemented in the same tile 104 are exchanged via the memory of that tile, while activations moving between tiles 104 are sent through the photonic structure. In embodiments where each neural node is implemented by a respective tile 104 of an electro-photonic network, the tile network topology will closely reflect the ML model (or ML graph topology).
[0055] As activations are streamed from an originating tile to a destination tile through the electro-optical network, the address of the destination tile for each output activation is typically determined by the electronic message router 110 of the originating tile of the activation, for example, based on the path setup during initialization of the ML model. The address can be provided as packet data as part of a message that carries the output activations as a payload. In one embodiment, the address is in the header of the message, and in another embodiment, the address is encoded into the message itself (as part of the payload). The address contained in the message is used by message routers along the path through the electro-optical network to route the output activations to the destination tile. In some embodiments, as described above, a relative addressing scheme is employed.
[0056] In a typical ML model, the number of weights is much larger than the number of activations. For current ML workloads, the ratio of the number of weights to the number of activations is typically in the range of about 20 to 1 to about 60 to 1. Therefore, even if activations are streamed from tile to tile, keeping the weights stationary saves a lot of energy. In addition, in some beneficial embodiments, L2SRAM 114 is physically arranged very close to DNN 106 and tensor engine 108, which reduces the interconnect distance through which the weights travel locally within each tile, thereby reducing the capacitance associated with the relevant interconnect length, and reducing the energy loss associated with such interconnect capacitance. In addition, it is apparent from the above description that activations streamed from an originating node in one tile to a destination node in another tile (e.g., over a relatively long distance) primarily pass through an optical path provided by PIC 102, while activations streamed within the same tile (e.g., over a relatively short distance) use an electrical path provided by EIC 101. In this way, the reliance on electrical interconnects for long-distance data movement is virtually eliminated, which significantly reduces the energy consumption associated with electrical interconnect capacitance. In other words, by streaming activations from tile to tile in a photonic manner, significant power savings can be achieved. In summary, using the ML processor or ML accelerator described herein, data movement is partially minimized (in the case of weights) and partially relies on power-efficient tile-to-tile data transfer via photonic connections (in the case of activations). In addition to providing power savings benefits, using photonic fabrics for data transfer can also reduce latency and provide higher bandwidth.
[0057] Figure 3 is a diagram showing an example tensor engine 108 according to some embodiments, which may be used in Figure 2AThe tensor engines 108 are used within the electronic processing element 104 shown. Each tensor engine 108 can be assigned to perform a portion of the inference calculation of a specific machine learning model implemented, for example, by the SIP 100 used as an ML processor. The tensor engines 108 in different processing elements 104 of the ML processor 100 can perform machine learning tasks in parallel or sequentially. The ML calculations of the ML processor can be performed in one or more tensor engines 108, thereby forming a data flow between the tensor engines.
[0058] Various implementations of the tensor engine 108 may be used without departing from the scope of the present application. In one example, Figure 3 As shown, the tensor engine 108 includes an instruction sequencer 300, a register bank 302, a multiplexer 304, a Ra register 306, a Rb register 308, an arithmetic logic unit (ALU) 310, an adder 312, and an Rg register 314. The tensor engine 108 uses the instruction sequencer 300 to perform register write, accumulation, and register read operations in a manner known in the art. For example, the tensor engine 108 can write two values to the Ra register 306 and the Rb register 308, accumulate them with the help of the ALU 310, and save the result in the Rg register 314. Thereafter, another two values are written to the Ra register 306 and the Rb register 308, accumulated with the help of the ALU 310, read from the ALU 310, and added to the previous contents in the Rg register 314 and written to the Rg register 314. This routine may be repeated again, for example, up to 32 times, to generate a 32-bit output from each output register of the tensor engine 108 .
[0059] The tensor engine 108 is a single instruction multiple data (SIMD) processor using an instruction set that, in some embodiments, may be specifically designed for executing machine learning algorithms. Although data movement between different tensor engines 108 can be performed electronically or photonically, in a preferred embodiment, data movement between tensor engines 108 in different processing elements 104 is performed photonically, while data movement within the same tensor engine 108 is performed electronically. The tensor engine includes an interface to the local port 206 of the message router 110, which allows it to communicate with the PIC 102.
[0060] Figure 4 is a diagram showing an example DNN 106 according to some embodiments, which can be used to Figure 2A The electronic processing unit 104 is shown. Figure 4The example DNN of SIP 100 is described with respect to DNN 106 of SIP 100, but for some embodiments, the illustrated DNN may form part of hardware implemented in other environments (e.g., as a component of other types of processors). As used herein, the DNN may also be referred to as a neural compute engine (NCE) or a dot product engine. Depending on the embodiment, the DNN may be implemented by one or more portions of hardware circuitry that is configured to generate (e.g., calculate) one or more dot products (e.g., dot product values) based on at least two vectors. For example, the at least two vectors may be composed of a set of weight values and an operand value (e.g., an activation value).
[0061] Reference now Figure 4 , shows an embodiment of a portion of a DNN 106 located in an EIC 101 of a SIP (e.g., an ML processor) 100. The DNN 106 is essentially a combinatorial tree of multipliers and adders that computes the dot product of a set of weights and operands (e.g., activations from previous nodes) in hardware without latching data during any operation. As a result, the DNN 106 has very few registers and consumes significantly less power by dedicating only certain predetermined routines without performing register transfer operations and without latching data. In the DNN 106, preloaded weights are retrieved from the L2SRAM 114, and the DNN sequencer 400 provides the weights (e.g., weight values) and operands (e.g., operand values) to the top of the combinatorial tree. For example, weights W1, W2, W3, and W4 are provided to the top of the tree, as Figure 4 Operands (eg, activations from previous nodes) X1, X2, X3, and X4 are also provided to the top of the tree.
[0062] In some embodiments, each weight W1, W2, W3, and W4 and each operand X1, X2, X3, and X4 consists of 8 bits. The weights W1, W2, W3, and W4 are multiplied with the corresponding operands X1, X2, X3, and X4. The result of each operation is a 16-bit product. Figure 4 As shown, the two 16-bit products are added to form a 17-bit sum. At the next level, the two 17-bit products are added to form an 18-bit sum, and so on for further levels of addition. The calculated dot products are then added to a 32-bit accumulator so that a sequence of dot products can be accumulated without loss of precision. A pipeline implementation of the combinatorial tree can be constructed to reclock the data as it flows through the combinatorial tree. In some embodiments, seven pipeline stages are used. As described above, DNN106 saves significant power during dot product calculations. For comparison, using 7nm CMOS Fin-FETs, Figure 3 A single dot product calculation performed by the tensor engine 108 in may cost about 250 femtojoules (250 fJ) of energy. Figure 4 A single dot product computation performed by the DNN 106 in may only cost approximately 130 femtojoules (130fJ) of energy.
[0063] about Fig.12 More information about various embodiments of DNNs is shown and described.
[0064] Figure 5A It is a diagram showing an example implementation of the photonic components and electronic components of an electro-photonic network according to some embodiments in a side view. EIC (e.g., ASIC) 101 is shown as being located on PIC 102. Indicated are multiple tiles (processing elements) 104A, 104B, 104C and 104D in the EIC 101, together with the optical link between the adjacent tiles implemented using the waveguide 500AB, 500BC, 500CD in the PIC 102 below. Also shown are EO and OE interface components associated with the optical link between the tiles and between them. As shown in the figure, the EO interface components include modulator drivers 502A, 502B, 502C that are respectively located in EIC tiles 104A, 104B, 104C, together with modulators (e.g., EAM) 504A, 504B, 504C that are located directly below corresponding modulator drivers 502A, 502B, 502C in PIC 102. The OE interface components include TIAs 506B, 506C, 506D located in the EIC tiles 104B, 104C, 104D, respectively, and photodetectors (PDs) 508B, 508C, 508D located directly below the corresponding TIAs 506B, 506C, 506D in the PIC 102. Optical waveguides 500AB, 500BC, 500CD each connect an optical modulator 504A, 504B, 504C associated with one tile to a photodetector 508B, 508C, 508D associated with an adjacent tile, thereby providing an optical link in the PIC 102 for intra-chip communication between corresponding pairs of tiles within the ML processor 100.
[0065] Avoiding lateral displacement between the PIC-side and EIC-side components of the EO and OE interfaces (e.g., by placing the modulator driver directly above the modulator and the TIA directly above the photodetector) minimizes the distance between the components in each interface; in some embodiments, the distance between the modulator and its associated driver and between the photodetector and its associated TIA is minimized so that the speed of signal conversion between the electrical and optical domains (and vice versa) is maximized. Typically, the spacing is less than 200 μm, although the spacing may vary depending on the nature of the embodiment, the configuration of the OE and EO interface components, the requirements and architecture of the system, the temperature of the system, etc. In one embodiment, the placement of the PIC-side and EIC-side components in close association has beneficial thermal properties for the PIC-side components. Since the EIC side is a heat sink that can radiate heat to the PIC-side components, this in turn can cause improved thermal stability of the components on the PIC side due to the increased heat. Minimizing the distance between the PIC-side and EIC-side components in turn minimizes the power consumption of signal conversion between the optical and electrical domains, which is important for the overall power efficiency of data movement on the hybrid electro-optical network described herein. Furthermore, in some examples, this is beneficial when there is heat generated from components in the EIC (such as a modulator driver or other functional blocks) that are placed directly above or in close association with the EAM on the PIC. In this case, thermal radiation from the EIC side can increase the operating temperature of the EAM or otherwise promote thermal stability in an enhanced manner, and / or bring the EAM closer to peak efficiency or closer to an optimal operating range.
[0066] Figure 5A Also shown is an optical coupling structure that can be used to establish a fiber optic connection between the SIP (or ML processor) 100 and other devices. The optical fiber can be coupled to the PIC 102 using edge coupling. Alternatively, as shown in FIG. Figure 5A As depicted, an optical fiber 133 may be attached to the PIC 102 via a FAU 132, and light may be coupled between the FAU 132 and the PIC 102 using a grating coupler 520 formed in the PIC 102 just below the FAU 132. The optical fiber 133 may be connected to an off-chip laser source, or to a FAU of another SIP (ML processor) that provides optical input to the PIC 102. Light received via the grating coupler 520 may then be supplied to the modulators 506A, 506B, 506C in the PIC 102 via an optical waveguide. Figure 5A In the cross-sectional view of FIG. 5 , only the waveguide 522 of the EAM 506A is visible, but in FIG. Figure 5B In the top view of FIG. 5 , the optical connections between the grating coupler 520 and all EAMs (including EAMs 506B and 506C) can be seen.
[0067] Figure 5B FIG. 1 is a top view showing a method according to some embodiments of the present invention. Figure 5A 104B. An illustration of PIC 102 for an example electro-photonic network of EIC 101. The position of tiles of EIC 101 relative to PIC 102 is indicated by dashed lines. For each pair of adjacent tiles (whether they are horizontally or vertically adjacent), PIC 102 provides two unidirectional photonic links, each of which includes an optical modulator (defining the input end of the photonic link), a photodetector (defining the output end of the photonic link), and a waveguide directly connecting the modulator to the photodetector. For example, waveguide 500AB connects modulator 504A associated with tile 104A to photodetector 508B associated with tile 104B. The two photonic links between any given pair of tiles are oriented in opposite directions and thereby collectively form a bidirectional photonic channel. Note that although Figure 1B Only one unidirectional photon link in each direction is shown, but a photon channel may typically include multiple links in each direction (e.g., grouped together) to increase the bandwidth for data transfer across the channel. Within a tile (or more specifically, within a router within a tile), a photon channel interface to a bidirectional photon channel in a PIC includes an optical driver for each outgoing photon link of the bidirectional channel (with which the router engages at the input end of the link) and a TIA (and associated line system) for each incoming photon link of the bidirectional photon channel (with which the router engages at the output end of the link).
[0068] PIC 102 receives an optical carrier signal from a laser source. In some embodiments (not shown), the light source is integrated in the PIC. In other embodiments, the light source is implemented outside the PIC, and light is provided to the PIC, for example, via edge coupling or via an optical fiber connected to FAU132 and a grating coupler in the PIC. From FAU 132 and the grating coupler, or any other input point of the PIC, one or more optical input waveguides guide the carrier signal to an optical modulator of a bidirectional photon channel, where the carrier signal is modulated to give a corresponding message to the optical signal for transmission via the photon channel. In some embodiments, as shown in the figure, the carrier signal is divided between multiple optical input waveguides (e.g., for each row of tiles in the EIC, one waveguide for each photon link pointing to the north, east, west and south). Along each input waveguide, additional splitters can branch light in turn to input to different modulators. For example, as depicted, waveguide 532 provides a carrier signal to splitter 534A in tile 104A, where light is split between an optical path to modulator 504A of that tile and another optical path that directs the light to splitter 534B in tile 104B via a continuation of waveguide 532. From splitter 534B in tile 104B, the carrier light is provided along one optical path to modulator 504B of tile 104B and along another optical path via a further continuation of waveguide 532 to modulator 504C in tile 104C.
[0069] like Figure 5A and Figure 5B As shown, in one embodiment, all or most tile-to-tile communications (e.g., streaming of activations from a first node in a first tile to a second node in a second tile) are performed using photonic channels in the PIC 102 by employing modulators (e.g., EAMs) and photodetectors in the PIC 102 (which are placed directly below their respective modulator drivers and TIAs on the EIC 101). The modulators and PDs are optically connected, for example, via waveguides in the PIC 102. Thus, in one embodiment, all or most tile-to-tile communications are not performed electrically within the EIC 101, but are performed optically using photonic structures provided by the PIC 102.
[0070] As described above, the electro-photonic networks of multiple SIPs described herein can be connected into a larger electro-photonic network via optical couplers on the PIC and optical transmission media between optical couplers of different SIPs. The transmission medium can be an optical fiber, such as a multi-fiber terminal push (MTP) ribbon fiber connection (up to 20 meters). Other transmission media are also possible, such as integrated optical waveguides (e.g., in separate PICs configured to provide optical interconnect structures between SIPs) or free space transmission (e.g., using imaging optics to couple light from the output of one PIC to the input of another PIC). In the case of using a grating coupler to capture light, the grating coupler can be configured so that the light leaves the PIC at a predefined angle associated with the angle of the optical fiber so as to minimize the loss of light. In other embodiments, various types of waveguides are possible, including diffraction gratings, such as echelle gratings, etc. Within such a multi-package system, messages between tiles of all EICs can be exchanged via the photonic structures implemented through the PICs and the optical interconnects between them, regardless of whether the messages are sent on-chip / intra-chip (meaning within a single SIP) or off-chip / inter-chip (meaning from a tile in one SIP to a tile in another SIP). However, according to various embodiments described herein, on-chip and inter-chip optical communication may differ in that on-chip communication is typically performed using a single wavelength of light, while inter-chip communication between different SIPs in a larger system (e.g., different ML processors in an ML accelerator) is typically performed using wavelength division multiplexing (WDM), which helps reduce the number of fiber connections required between different chips (e.g., different ML processors). Note that in some embodiments, multiple wavelengths are also used for on-chip optical communication, and conversely, single wavelength communication can in principle also be used for off-chip communication.
[0071] Fig. 6Ais a diagram showing an example group of photonic links connecting two processing elements within a SIP according to some embodiments. For intra-chip optical communications, an optical engine 600, which may be an on-chip or off-chip laser source, outputs carrier light at a single wavelength λa1. A splitter tree 602 may divide light having wavelength λa1 between multiple optical paths 604 (only one is shown) leading to modulators associated with different tiles and photonic channels. Along each of these paths 604, light may be further split between groups of modulators 610, 612, 614, 616 associated with groups of unidirectional photonic links in the same direction, which are combined together to form a combined group corresponding to a unidirectional photonic channel from one tile to another in the SIP; two such unidirectional photonic channels between the same pair of tiles of the SIP form an intra-chip bidirectional photonic channel. Modulators 610, 612, 614, 616 (eg, EAMs) modulate light having a single wavelength λa1 and transmit the modulated light via respective waveguides 620, 622, 624, 626 to photodetectors 630, 632, 634, 636 located in different tiles of the same chip.
[0072] Figure 6B is a diagram showing an example system of two SIPs 100A, 100B using WDM for inter-chip optical communication in accordance with some embodiments. In the depicted example, the PICs of the SIPs 100A, 100B are connected via a grating coupler 640 in the first PIC, a first optical fiber 641, an optical fiber connector 642, a second optical fiber 643, and a grating coupler 645 in the second PIC. In the first SIP 100A, an optical engine 650, which may be an on-chip or off-chip laser source, provides light at multiple wavelengths (e.g., between 2 and 16 wavelengths), such as the four wavelengths λb1, λb2, λb3, λb4 as shown, to the PIC. A splitter tree 602 (e.g., similar to Fig. 6A The splitter tree of FIG. 100A can divide light at multiple wavelengths between multiple optical paths 652 (only one is shown) that lead to different tiles of the SIP 100A and / or optical modulators associated with different photon channel interfaces within a tile. For example, carrier light at wavelengths λb1, λb2, λb3, λb4 can be provided to multiple peripheral photon channel interfaces (such as Figure 2B An optical modulator associated with interface 214) as shown.
[0073] Along each of these paths 652, a demultiplexer 654 separates the wavelengths λb1, λb2, λb3, λb4 between different corresponding modulators (e.g., EAMs) 662, 664, 666, 668. The modulators 662, 664, 666, 668 modulate the carrier light at the wavelengths λb1, λb2, λb3, λb4 and provide modulated optical signals having the corresponding wavelengths λb1, λb2, λb3, λb4 to a WDM multiplexer 680 on optical links 672A, 674A, 676A, 678A. The multiplexed output of the WDM multiplexer 680 (containing four data streams, each encoded on a separate wavelength λb1, λb2, λb3, λb4) is provided on a single waveguide to a grating coupler 640, where the multiplexed modulated optical signal is coupled off-chip to an optical fiber 641.
[0074] At the fiber connector 642, the multiplexed modulated optical signal is coupled from the optical fiber 641 into the second optical fiber 643. The second optical fiber 643 couples the multiplexed modulated optical signal into the PIC of the second SIP 100B via a grating coupler 645. In some other embodiments, edge-coupled optical fibers may be used instead of or in addition to the FAU and grating coupler. In the SIP 100B, the demultiplexer 682 then demultiplexes the multiplexed modulated optical signal to output four separate modulated signals having wavelengths λb1, λb2, λb3, λb4. These four signals are provided to the photodiodes 692, 694, 696, 698 in the SIP 100B via corresponding optical waveguides 672B, 674B, 676B, 678B. In general, the modulators 662, 664, 666, 668, waveguides 672A, 674A, 676A, 678A, multiplexer 680 and grating coupler 640 in the PIC of SIP 100A, the fiber optic connections (641, 642, 643) between SIPs 100A and 100B, and the grating coupler 645, demultiplexer 682, waveguides 672B, 674B, 676B, 678B and photodetectors 692, 694, 696, 698 in the PIC of SIP 100B form unidirectional photon channels between tiles of different SIPs; two such unidirectional photon channels between the same pair of tiles of SIPs form bidirectional photon channels between chips.
[0075] Although the embodiments discussed above are directed to photonic channels showing four optical links in one direction and a WDM multiplexer receiving four different wavelengths, in other embodiments, two or more optical links and a WDM multiplexer receiving two or more different wavelengths may be used. Thus, the demultiplexer will output two or more different wavelengths corresponding to these alternative embodiments.
[0076] Using intra-chip and inter-chip photonic channels, for example, as described above, typically including one or more links in each direction, processing elements (tiles) in (one or more) EICs of one or more SIPs can be connected into an electro-optical network. The resulting network topology typically depends on the selection of tile pairs that are directly connected via associated photonic channels; refer to Figures 7 to 9 Various example topologies are described. Note that while the present disclosure generally refers to bidirectional photon channels, which give rise to network structures that provide greater flexibility for implementing ML and other computational models than unidirectional photon channels, electro-photonic networks can in principle also be formed from unidirectional photon channels, and such networks can retain many of the benefits discussed herein (e.g., power savings due to photonic data delivery over longer distances).
[0077] Figure 7 is a diagram showing an example electro-optical network 700 having a quadrilateral mesh topology according to some embodiments. This topology is also for Figure 1B The depicted SIP 100 is shown. In the depicted example, the network includes sixteen tiles 104 (constituting network nodes) arranged in a quadrilateral (e.g., rectangular, as shown) grid and interconnected by photon channels 702 so that the output of each tile can be provided as an input to any of its immediately adjacent tiles. In alternative embodiments, other network topologies can be supported to provide richer or sparser connections, for example, based on the needs of a particular application. For example, additional photon channels can be added to skip network nodes and provide direct paths to non-adjacent nodes (i.e., nodes that are two or more hops away in the rectangular grid).
[0078] Figure 8 is a diagram illustrating an example electro-optical network 800 having a wrapped torus mesh topology according to some embodiments. Figure 7 Like the network 700 in FIG. 1 , the network 800 includes sixteen tiles 104 arranged in a rectangular grid. In this topology, the photon channels 802, 804 between the tiles 104 are configured so that the output of each tile 104 can be provided to the input of any of its immediately adjacent tiles via the photon channel 802 ( Figure 7), and in addition, each tile along the periphery of the array can provide its output directly to tiles on the opposite edge of the grid via photon channel 804, i.e., for tiles arranged along the vertical edges, distant tiles in the same row, and for tiles arranged along the horizontal edges, distant tiles in the same column. The resulting wrapped torus has the beneficial effect of significantly reducing the number of tile-to-tile hops required for communication between tiles on opposite sides of the network. For example, without photon channel 804, a message sent from a tile on the left edge of the network to a tile on the right edge of the network would traverse an entire row of tiles in the network.
[0079] In some embodiments, a wrapped torus topology is physically implemented by combining optical links at the edge of the chip into grating couplers that match fiber attachment units, or alternatively into edge couplers, providing connections for an array of optical fibers assembled into ribbon fibers. Ribbon fibers can be designed to allow arbitrary topologies of optical links to be constructed using patch fibers. Patch fibers allow connections from one side of the PIC to wrap around and form a wrapped torus. Alternatively, for the use of optical fibers, the photon channels 804 connecting tiles on opposite edges can also be implemented as optical waveguides within the PIC. Optionally, in order to avoid waveguide crossing between photon channels 804, the waveguides connecting tiles on opposite ends of the same row can be implemented in a layer separate from the waveguides connecting tiles on opposite ends of the same column.
[0080] Fig. 9 is a diagram showing an example electro-photonic network 900 having a scalable wrapped torus mesh topology in accordance with some embodiments. In this case, four SIPs 100 (e.g., implementing an ML accelerator having four ML processors) (each SIP including sixteen tiles 104 in a rectangular grid in its EIC) are themselves arranged in a two-by-two grid, resulting in an overall arrangement of sixty-four tiles 104 in an eight-by-eight grid. As shown, in this configuration, tiles 104 that are immediately adjacent to each other within the same SIP 100 or in two adjacent SIPs 100 are connected by photonic channels (e.g., implemented by waveguides in the PICs for intra-chip photonic channels 902 and optical fibers for inter-chip photonic channels 904). In addition, tiles 104 along the periphery of the eight-by-eight grid are interconnected (by inter-chip photonic channels 906) as a wrapped torus, similar to Figure 8, such that the output of each tile 104 in one SIP 100 located at the edge of the eight-by-eight grid can be provided as an input to a tile of another SIP on the opposite edge of the grid. Beneficially, the inter-chip photonic channels 904, 906 seamlessly integrate the tiles 104 of all SIPs 100 into the larger electro-photonic network 900. As will be appreciated, the number of SIPs 100 connected into a single larger network can be different than the four depicted, and can be directly increased without incurring significant energy losses, thereby taking advantage of the energy efficiency of photonic communications over long distances. In addition, the scalable wrapped torus topology reduces the number of tile-to-tile hops that would otherwise be required to communicate between tiles on the far ends of the grid.
[0081] Figures 7 to 9 Various non-limiting examples of possible network topologies are shown. To provide yet another example, the photon channel may also be configured to connect tiles within and / or across one or more SIPs together into a three-dimensional cubic grid topology. In such an embodiment, the tiles may still be arranged in a single plane (i.e., geometrically two-dimensional), but have the characteristics of a three-dimensional network topology by virtue of the connections between network nodes. Note that Figure 8 and Fig. 9 The depicted wrapped torus network is also an example of a configuration that is geometrically two-dimensional and topologically three-dimensional.
[0082] Note that in an ML accelerator system according to various embodiments described herein, several techniques can be used to support power-efficient execution of ML models using photonic structures. For ML models, both the mode and schedule of communication are known at compile time, which provides opportunities for compiler-guided optimization of photonic structures. For example, based on the ML model loaded into the ML processor or accelerator, unused photonic channels and optical links in the photonic structure can be turned off to save power. For example, a bidirectional photonic channel consisting of eight links (four in each direction) can provide more bandwidth than a given model requires. Dynamic binding can be used to selectively activate and deactivate optical links in the channel based on a schedule developed by the compiler according to the needs of the ML model, thereby further reducing power consumption. A prerequisite for implementing this algorithm is that the EO and OE interfaces can be inactive (quiescent) and restarted for a few bits of time, which is achieved by selecting the master link from each channel at boot time using a negotiation protocol. The master link of the channel is never shut down. The transmit clock on the master link is used as the transmit clock on all slave links. At the receiver, the master link performs clock data recovery on the incoming bit stream and reconstructs the transmitter clock. Then, the receive clock is distributed to other slave links in the channel. Since there is a delay in the transmission between the master and slave links at both the transmitter and the receiver, the clock skew must be accounted for in each receive link. This is achieved by performing a channel alignment phase during channel bring up.
[0083] The configurability of the photonic structure and its dynamic combining capabilities can be further used to provide fault tolerance to correct manufacturing defects caused by low-yield components. Modulators (such as EAMs) and photodiodes may have manufacturing defects, which cause them to have permanent defects. When the ML accelerator is initialized, each channel in the photonic structure is set up and calibrated so that it is ready to carry data. In normal operation, the links in the channels are combined together to form a group, which performs as a single logical channel. During the process of channel establishment, non-functional links can be omitted from the combined group and disabled. The channel establishment process is coordinated by a state machine in the port logic of each channel.
[0084] The channel establishment state machine proceeds in phases. In the first phase, the primary link is selected. The transmitter rotates through each link in the channel, transmitting a link establishment framing pattern. When a receiving channel successfully receives a link establishment pattern, it switches its link establishment to a "primary discovery" framing pattern. When a channel receives a "primary discovery" framing pattern, the corresponding transmitter will stop rotating through the link and wait for several "primary discovery" framing patterns to indicate that the primary selection is stable. If the channel stops receiving "primary discovery", it will revert to using the link establishment framing pattern. When the primary link is established in both directions, bonding can proceed. For each link that is not currently the primary link, the system transmits a "ready to bond" framing pattern. When the receiver correctly decodes the "ready to bond" framing pattern, it transmits a bonded framing pattern on the primary link and adds the receiving channel to the bonding group. This process continues until all non-primary links are dropped or added to the bonding group. At the completion of the bonding process, only those links that are fully functional will be included in the bonding group.
[0085] Fig.10 is a diagram showing conventional techniques for clock signal distribution within a processor. Conventional techniques for clock signal distribution within a processor consume a lot of energy. The clock tree is usually built hierarchically, starting with a root clock network that distributes the clock signals to regional clock networks. The typical topology is Fig.10 The "H-tree network" shown, where the clock segments are sequentially divided on equal length paths to distribute the clock signal to the regions of the processor. Buffers 1044 are used to regenerate the clock signal as the clock signal propagates over long distances across the processor. The paths through the H-tree are matched in length to maintain uniform clock skew between the arms of the H-tree network. Then, within the region, there are further levels of hierarchical clock splits to connect the clock signal down the regional level to the local lines 1046. Because the clock paths must be matched in length, the root clock network often has routing challenges and also consumes a lot of power in the regeneration buffers 1044.
[0086] Fig.11 is a diagram showing an example of using a photonic structure to distribute clock signals (such as a root clock signal) according to some embodiments. For some embodiments, the photonic structure described herein is used to carry clock signals within an ML processor (e.g., 100), such as the clock signal of a root clock network used by an ASIC (or other EIC, e.g., 101) of the ML processor. Now referring to Fig.11, since the master link is always transmitted on channels 1112, 1114, 1116, and 1118, the receiving tile can use the selection by the first master link 1130 to establish the clock source of the tile (e.g., 104). The clock signal is extracted from the incoming data stream on channels 1112, 1114, 1116, and 1118 by using a clock data recovery (CDR) line 1120 and fed into a jitter attenuation PLL 1124, which produces a low jitter clock signal 1126, which is then used for transmission on all other channels 1132, 1134, 1136, and 1138 in the tile, as well as divided down and used as a digital clock signal 1140 for tensor processors, DNNs, and message routers.
[0087] Thus, each tile establishes its own clock domain, and the electrical connections between tiles are timed using a symmetrical clock. This arrangement has several benefits over a digital global clock network: it saves power by eliminating the root clock network typical in a processor; it reduces the complexity of the clock network that must be routed around the processor and between several processors; and it reduces current spikes in the digital logic due to synchronized global clock transitions. The novel clock signal distribution mechanism of some embodiments extends between ML processors and within tiles of a single ML processor.
[0088] Although the concepts of the present invention are described herein using various embodiments, other embodiments of these concepts may be devised by one of ordinary skill in the art without departing from the scope of this application. For example, in one approach, the DNN itself may be a PNN (photonic neural network), where the neural network layers are implemented using photonics (e.g., using beam splitters and optical waveguides to perform MAC functions, and where intermediate output signals are optically combined for optical output). Furthermore, portions of the photonic network and / or photonic structure may be implemented using electrical connections in addition to or in lieu of optical connections, e.g., as a backup connection, or for use during testing or initialization of an ML accelerator.
[0089] In summary, according to various embodiments of the inventive concept, the present ML accelerator includes a novel hybrid electronic-photonic (or electro-photonic) NoC and a novel DNN, as well as an overall unique architecture for executing ML models and a novel clock timing scheme. The currently disclosed ML accelerator significantly reduces power consumption while increasing processing speed by reducing the power consumed by data movement and reducing energy consumption in MAC operations. Reducing the power consumed by data movement is achieved by maximizing data locality in each ML processor (e.g., by reducing data movement) and further by reducing energy loss when data movement is required. In addition, the novel clock timing scheme in the present ML accelerator results in additional power savings.
[0090] For some embodiments, the clock signal distribution described herein is implemented with respect to a line package (such as a SIP). For example, the line package may include an ASIC (e.g., 101) that includes multiple processing elements (e.g., multiple tiles) that include a photon channel interface. In addition, the line package may include multiple photon channels that connect the multiple processing elements to form at least a portion of an electro-optical network (e.g., network 130). According to various embodiments of the multiple photon channels, the multiple photon channels (e.g., channels 1112, 1114, 1116, 1118, 1132, 1134, 1136, 1138) are connected to individual processing elements (e.g., tile 104) in the multiple processing elements.
[0091] To facilitate clock signal distribution of embodiments, the individual processing element may select a master photon channel (e.g., one of channels 1112, 1114, 1116, 1118) for the individual processing element from the plurality of photon channels (e.g., via first main link 130). For some embodiments, the individual processing element selects the master photon channel during boot time of the individual processing element (e.g., using a negotiation protocol).
[0092] The individual processing element may select a set of slave photonic channels (e.g., channels 1132, 1134, 1136, 1138) for the individual processing element from the plurality of slave photonic channels. For some embodiments, the individual processing element selects the set of slave photonic channels during a boot time of the individual processing element (e.g., using a negotiation protocol). Additionally, for some embodiments, the individual processing element performs a channel alignment operation on the master photonic channel, the channel alignment operation being configured to adjust one or more settings of the individual processing element to compensate for clock skew.
[0093] The individual processing elements may extract the received clock signal from the selected master photon channel.For some embodiments, the individual processing elements extract the received clock signal from the master photon channel by performing clock data recovery (eg, via CDR line 1120) on the incoming bit stream received on the master photon channel.
[0094] The individual processing element may generate a local clock signal (e.g., digital clock signal 1140) based on the received clock signal, and the individual processing element (e.g., tile 104) uses the local clock signal to perform one or more local operations on the individual processing element. The local operation on the individual processing element may be performed by at least one of a tensor processor (e.g., 108), a DNN (e.g., 106), or a message router (e.g., 110) of the individual processing element (e.g., tile 104). Thus, the individual processing element may include a tensor processor (e.g., 108), and the tensor processor may operate based on the local clock signal. The individual processing element may include a message router, the message router includes a photon channel interface, and the message router may operate based on the local clock signal. The individual processing element may include a hardware circuit (e.g., DNN 106) for calculating a dot product between at least two vectors, and the hardware circuit may operate based on the local clock signal.
[0095] For some embodiments, an individual processing element (e.g., tile 104) includes a jitter attenuating phase-locked loop (PLL) (e.g., 1124), wherein the individual processing element uses the jitter attenuating PLL to generate a low-jitter clock signal based on a received clock signal. The individual processing element can generate a local clock signal based on the received clock signal by generating a local clock signal based on the low-jitter clock signal.
[0096] Ultimately, an individual processing element may transmit a distributed clock signal from a set of photonic channels (e.g., channels 1132, 1134, 1136, 1138) to a set of multiple processing elements connected to the individual processing element, where the distributed clock signal is generated based on a received clock signal (e.g., the output of CDR circuit 20). For example, an individual processing element may transmit a distributed clock signal by transmitting one or more outgoing bit streams to the set of processing elements, where the one or more outgoing bit streams are generated based on the distributed clock signal. In this manner, one or more processing elements that receive the outgoing bit streams (by way of the set of channels) may then independently extract the distributed clock signal from the outgoing bit streams.
[0097] For some embodiments, the individual processing element is connected to a second processing element (of a plurality of processing elements) that is configured to operate similarly to the individual processing element. For example, the second processing element may be connected to the individual processing element via an individual photon channel of a set of slave photon channels (of the individual processing element). The second processing element may select the individual photon channel as a second master photon channel of the second processing element. The second processing element may select a second set of slave photon channels for the second processing element from a second sub-multiple photon channels connected to the second processing element. The second processing element may extract a second received clock signal from the second master photon channel (e.g., from an outgoing bit stream received by the second processing element via the second master photon channel). The second processing element may generate a second local clock signal based on the second received clock signal, and the second processing element may use the second local clock signal to perform one or more local operations on the second processing element. Thereafter, the second processing element may transmit a second distributed clock signal to a second set of multiple processing elements connected to the second processing element via a second set of slave photon channels, wherein the second distributed clock signal is generated based on the second received clock signal.
[0098] Fig.12 1 is a diagram illustrating an example architecture 1200 of a DNN according to some embodiments. As shown, the architecture 1200 includes a plurality of memory devices including sixty-four SRAM memory banks 1210-1 to 1210-64 (hereinafter collectively referred to as SRAM memory banks 1210), a switch fabric 1220 (e.g., an SRAM fabric), a controller 1230, and a dot product unit array 1240, wherein each dot product unit includes a plurality of fused multiply-accumulate (FMA) units including an FMA unit 1250.
[0099] The architecture 1200 can perform compute-intensive functions, such as convolutions and matrix multiplications (which are commonly used in the inner loops of ML models), with minimal overhead per operation. The architecture 1200 can optimally perform specific compute-intensive functions with little overhead by removing the overhead associated with general programmability compared to conventional techniques (such as register transfer machines used to execute ML graphs). The architecture 1200 can provide a close association with the SRAM memory bank 1210 that feeds the switch fabric 1220, which can minimize the movement of data to math units (e.g., dot product unit array 1240) when exchanging values between math units. In this way, the architecture 1200 can provide a near-memory computing architecture that also saves power compared to conventional techniques.
[0100] According to some embodiments, architecture 1200 represents a hardware circuit of a DNN, where the hardware circuit includes a combination tree (e.g., Figure 4) and a sequencer (e.g., 400). The combinatorial tree may use a multiply-accumulate (MAC) unit to generate one or more dot product values based on input values received at a top level of the combinatorial tree, and the sequencer 1211 may be configured to rearrange the position of the input vector received from the memory device set 1210 in the switch structure 1220. In this manner, some embodiments may control the order of weight values and operand values provided as input vectors to the top level (e.g., the leftmost column in the 2-dimensional dot product array 1240). The combinatorial tree may include registers that are configured to maintain clock alignment of values flowing through the combinatorial tree. In Fig.12 12, clock alignment of the values is maintained relative to clock 1280 connected to dot product unit array 1240. As shown, each FMA of dot product unit array 1240 is connected to clock 1280.
[0101] The sequencer 1211 is operably coupled to a switch fabric 1220 or a memory device, which operably couples the combinatorial tree to a set of memory devices 1210, such as local SRAM memory devices, which in this example are shown as 64 memory banks, each providing a 32-bit vector. The set of memory devices may include SRAM memory banks 1210 or any other suitable local memory. The combinatorial tree may include an array of dot product cells, such as dot product cell array 1240. The input of each dot product cell of the array forms a top level of the combinatorial tree that receives an input value (e.g., a 32-bit vector).
[0102] A dot product unit array 1240, which can be configured to receive an input value (e.g., an input vector 1260) and generate one or more dot product values based on the input value received by the top level of the combinatorial tree, wherein a single dot product unit can include two or more of the MAC units. A single MAC unit of the dot product unit has at least one accumulator. In some embodiments, multiple accumulators are configured to accumulate partial dot product values generated by a single MAC unit as values flow through the single MAC unit. The MAC unit can include at least one FMA unit, such as FMA 1250.
[0103] For some embodiments, the sequencer 1211 is configured to provide vectors from the memory device set 1210 to the switch fabric 1220. In some embodiments, the weight values and operand values (e.g., activation values) are provided as a sequence of input vectors to the top level by the switch fabric 1220 according to a controller (e.g., controller logic) that controls the operation or configuration of the sequencer 1211, a clock, a tensor engine, or another component associated with the ASIC. For example, the controller 1230 may implement the sequencer 1211 in hardware, firmware, software, or a combination of all of these, wherein the sequence of input vectors (e.g., 1260) provided to the dot product unit array 1240 is determined by the controller 1230. Depending on the embodiment, the sequencer 1211 may determine the sequence of input vectors provided to the top level based on a parameter set, wherein the parameter set includes at least one of a stride value, a dilation value, or a kernel size value (e.g., which may be received or processed by the controller 1230).
[0104] The sequencer 1211 may be configured to generate an individual input vector (of an input vector sequence) by reading a plurality of weight values and a plurality of operand values read from a memory device set (e.g., an SRAM memory group 1210) during a single or multiple clock cycles, wherein the individual input vector includes a plurality of weight values and a plurality of operand values. According to various embodiments, the sequencer reads a plurality of weight values and a plurality of operand values for a first time, and no memory group conflict occurs during the sequencer read because it arranges the input vectors in the switching structure 1220 that provides input to the dot product engine 1240. In addition, in the case where the memory device set includes a memory group, the sequencer may read from each of the memory groups during a single clock cycle, although other clock-based schemes are also possible. For some embodiments, each successive read of the memory device set by the sequencer causes a new weight value and a new operand value to be read from the memory device set that were not read by the sequencer during the previous clock cycle. Specifically, the switching structure 1220 may read the weight value and the operand value (e.g., activation value) only once when the input data is arranged, so that there is no memory group conflict. By reading individual weight values and operand values only once, the number of times individual weight values or operand values are accessed from a set of memory devices is reduced.
[0105] The dot product unit array 1240 may include a data path that causes at least two weight values to be interchanged between adjacent rows of the array and propagates the weight values down the columns of the array. In addition, a single MAC unit of the dot product unit may be operably coupled to at least two adjacent MAC units of the dot product unit such that an output value generated by the single MAC unit is shared as an input value of each of the at least two adjacent MAC units.
[0106] Finally, after the sequence of input vectors has all been input to the top level of the combinatorial tree, logic (e.g., controller 1230) can cause a final set of dot product values to be generated for the weight values and operand values. For example, the final set of dot product values generated by the logic can be constructed from the values stored in the accumulators of the MAC units of the combinatorial tree. In addition, the logic is configured to store the final set of dot product values to the memory device set 1210. Alternatively, the system can be programmed to execute deterministically and automatically write the accumulator values back to the memory after a given number of computation cycles. In this way, the stored final set of dot product values can be used to construct another input vector for the dot product unit array.
[0107] During operation, one or more weight values and one or more operand values (e.g., activation values) may be stored on the SRAM memory bank 1210. For some embodiments, the SRAM memory bank 1210 includes 64 groups of 32-byte wide SRAM memories. The SRAM memory bank 1210 may feed the input of the switch structure 1220. The switch structure 1220 may use a sequencer 1211 to reorganize the data read (from the SRAM memory bank 1210) so that each consecutive memory read causes a new weight value or operand value to be delivered to the dot product unit array 1240 for performing a computational function (e.g., convolution or matrix multiplication). The switch structure 1220 may arrange the data read from the SRAM memory bank 1210 (e.g., using a controller, a sequencer, or a combination of both) according to one or more parameters of the ongoing operation (e.g., convolution or matrix multiplication) and the computational step currently being performed. Thus, the parameters may drive the order in which the operand values (e.g., activation values) are applied to the weight values. For example, the parameters for a two-dimensional (2D) convolution operation may include a step size, an expansion, or a kernel size. The output of the switch fabric 1220 may drive the input of the dot product unit array 1240 with the weight values and operand values read from the switch fabric 1220. As the values are read from the SRAM memory bank 1210, the pipeline of the switch fabric 1220 is filled with values and these values begin to clock out of the switch fabric 1220 into the dot product unit array 1240.
[0108] For some embodiments, during each clock cycle, weight values and operand values (e.g., activation values) are read by the switch fabric 1220 as a vector set, traversed through the switch fabric 1220 to the dot product unit array 1240, and arranged as the vector set traverses the switch fabric 1220. For convolution operations, the vector set read from the SRAM memory bank 1210 may be organized as patches (e.g., 8x8 patches) of channel vectors. For some embodiments, the switch fabric 1220 avoids bank conflicts when reading from the SRAM memory bank 1210 by arranging the patches in the memory. In this way, the switch fabric 1220 is able to read from all SRAM memory banks 1210 at each clock cycle.
[0109] For some embodiments, each dot product unit of dot product unit array 1240 is thirty-two FMA units deep. In addition, for some embodiments, dot product unit array 1240 includes 64 rows by 32 columns of dot product units. During operation, dot product unit array 1240 can perform 65,536 FMA operations per clock cycle. As shown, for some embodiments, each FMA unit (e.g., FMA unit 1250) in dot product unit array 1240 includes an accumulator 1252. In this example, it is shown as a pair of accumulators (ACC0 and ACC1) that are configured to accumulate partial product values as weight values and operand values (e.g., activation values) flow through the FMA unit. In other embodiments, other accumulator schemes may be used. For example, an FMA unit may receive two 8-bit values as input and generate a 32-bit dot product value (stored in the accumulator 1252 of the FMA unit). In various embodiments, the output accumulator of the FMA remains stationary while the value is streamed through the FMA unit. By keeping the output accumulator quiescent, various embodiments avoid significant energy consumption because the arithmetic logic unit and memory read and write operations associated with each FMA cycle and each calculation performed by the FMA unit are not required. For example, the energy consumption of FMA can be 130 fJ.
[0110] For some embodiments, each level of the dot product cell array 1240 includes a set of connections between adjacent FMA units, and a set of connections between adjacent FMA units between dot product values, which can allow weight values to be scrambled between FMAs. The scrambling of weight values can be controlled by one or more parameters (e.g., via the controller 1230), such as convolution parameters or matrix multiplication dimensions. In various embodiments, the dot product cell array 1240 is combinatorial, which enables the dot product cell array 1240 to reduce or avoid the energy cost of latching data in registers at each level of calculation within the dot product cell array 1240. For some embodiments, the dot product cell array 1240 includes a set of pipeline registers (e.g., a small number of pipeline registers) that are used to keep the data flow in the dot product cell array 1240 aligned with the clock signal driving the dot product cell array 1240. In addition, for some embodiments, the dot product cell array 1240 includes a set of data paths that allow weight values to be interchanged between rows of the dot product cell array 1240 and allow weight values to be propagated down through the columns of the dot product cell array 1240, which facilitates supplying the correct weight value to each operand value (e.g., activation value) as the operand value is streamed through the dot product cell array 1240. After all weight values and operand values have been transmitted through the dot product cell array 1240, the calculation of the weight values and operand values by the dot product cell array 1240 can be considered complete, and the result values in the accumulator of each FMA unit in the dot product cell array 1240 constitute the final dot product value set. These final dot product values can be written back (e.g., stored) to the SRAM memory group 1210, and then, during future subsequent calculations of the dot product cell array 1240, one or more of those stored values can be used as input to the dot product cell array 1240.
[0111] Fig.13 1300 is a flowchart illustrating an example method 1300 for operating a DNN according to some embodiments. It should be understood that the example methods described herein may be performed by a standalone DNN or by a device (such as ML processor 100) that includes a DNN (e.g., DNN 106). In addition, the example methods described herein may be implemented in the form of executable instructions stored on a machine-readable medium or in the form of electronic circuit systems (e.g., logic) that may cause the DNN to operate according to method 1300. For example, Fig.13 The operations of method 1300 may be represented by executable instructions that, when executed by a controller of the DNN, cause the DNN to perform according to method 1300. Depending on the embodiment, the operations of the example methods described herein may be repeated in different ways or involve intervening operations not shown. Although the operations of the example methods may be depicted and described in a particular order, the order in which the operations are performed may vary between embodiments, including performing particular operations in parallel.
[0112] Reference now Fig.13 , at operation 1302, a sequencer of the DNN reads a first plurality of weight values and a first plurality of operand values from a memory device set operably coupled to the sequencer. At operation 1304, a first input vector is received from the sequencer at a top level of a combinatorial tree of the DNN, wherein the first input vector includes the first plurality of weight values and the first plurality of operand values read at operation 1302. During operation 1306, the combinatorial tree processes the first input vector, wherein the combinatorial tree is configured to generate one or more dot product values based on values received through the top level using a MAC unit. At operation 1308, the sequencer reads a second plurality of weight values and a second plurality of operand values from the memory device set. At operation 1310, a second input vector is received from the sequencer at the top level of the combinatorial tree, wherein the second input vector includes a second plurality of weight values and a second plurality of operand values. During operation 1312, the combinatorial tree processes the second input vector, wherein the first input vector and the second input vector are part of a sequence of input vectors to be received by the top level of the combinatorial tree. After the top level of the combination tree has received the sequence of input vectors, at operation 1314, the DNN constructs a final set of dot product values from the values stored in the accumulators of the MAC units of the combination tree.
[0113] Depending on the size of the accumulator, this may include multiple steps. In some embodiments, the accumulator maintains 32 bits. Once enough input vectors flow through the dot product array to fill the accumulator, for example, at the top level of the combinatorial tree (or elsewhere in the dot product array), the values in the accumulators of the MAC units of the combinatorial tree (e.g., the accumulators of the columns of the filled dot product unit array) may be written back to the set of memory devices used in operation 1302 in operation 1316. Thereafter, the process may optionally be repeated. Thus, the accumulator output may be used by one embodiment as input for another processing cycle, thereby avoiding the use of an arithmetic logic unit or any memory access required to process or store the intermediate or final results of any calculations performed by the rows or columns of the combinatorial tree.
[0114] Fig.141400 is a flowchart illustrating an example method 1400 for clock signal distribution using an electro-optical network according to some embodiments. It should be understood that the example methods described herein can be performed by individual processing elements (e.g., tile 104) of an ML processor (e.g., 100). In addition, the example methods described herein can be implemented in the form of executable instructions stored on a machine-readable medium or in the form of an electronic circuit system (e.g., logic), which can cause the individual processing elements to operate according to the method 1400. Depending on the embodiment, the operations of the example methods described herein can be repeated in different ways or involve intervening operations not shown. Although the operations of the example methods can be depicted and described in a particular order, the order in which the operations are performed can vary between embodiments, including performing certain operations in parallel.
[0115] Reference now Fig.14 , at operation 1402, an individual processing element of the ML processor performs a channel alignment operation on a master photon channel, wherein the channel alignment operation is configured to adjust one or more settings of the individual processing element to compensate for clock skew. At operation 1404, the individual processing element selects a master photon channel from a plurality of photon channels, wherein the master photon channel is selected for the individual processing element. For some embodiments, the plurality of processing elements are part of an ASIC of a circuit package, and the plurality of processing elements are connected together through a plurality of photon channels to form at least a portion of an electro-optical network within the circuit package. During operation 1406, the individual processing element selects a slave photon channel set from a plurality of photon channels, wherein the slave photon channel set is selected for the individual processing element. At operation 1408, the individual processing element extracts a received clock signal from the master photon channel. Based on the received clock signal (extracted at operation 1408), the individual processing element generates a local clock signal, and the individual processing element uses the local clock signal to perform one or more local operations on the individual processing element. Based on the received clock signal (generated at operation 1410), at operation 1412, the individual processing element generates a distributed clock signal. At operation 1414, the individual processing element transmits the distributed clock signal to a set of multiple processing elements connected to the individual processing element by the set of slave photonic channels. For various embodiments, operation 1414 includes converting the distributed clock signal from an electrical form to an optical form using the photonic channel interface of the individual processing element. The distributed clock signal is suitable for transmission across the set of slave photonic channels. Further, for some embodiments, operation 1414 includes converting the distributed clock signal from an optical form to an electrical form at the set of multiple processing elements, wherein the distributed clock signal is suitable for use by the set of multiple processing elements as an associated local clock signal.
[0116] Fig.151 is a flowchart illustrating an example method 1500 for delivering messages within an electro-optical network (e.g., network 130 of SIP 100) according to some embodiments. At operation 1502, a message is generated at one of the processing elements 104 of the network (also referred to herein as a "first processing element" or "source tile"). The message is a packet of binary information including a series of 1s and 0s. A portion of the data included in the packet may optionally include data from a local memory of the tile, such as weights or activations. This is generally referred to as the payload of the packet. Another portion of the data included in the packet is a header, which may include routing information associated with a destination tile, such as an address. Alternatively, routing information may also be provided as part of the payload, for example, in a first portion separate from a second portion of the payload carrying data (e.g., weights or activations). Thus, the routing information in the packet may specify another processing element of the network (also referred to herein as a "second processing element" or "destination tile") as the destination of the message. Based on the routing information, a path from the source tile to the destination tile through the network may be determined. Unless the source tile and the destination tile are directly connected via a bidirectional photon channel of the network, the path includes one or more intervening processing elements (also referred to herein as "third processing elements" or "intervening tiles") on the route from the source tile to the destination tile. In some embodiments, the path is determined jointly by the routers of the source tile and the intervening tiles (by modifying at least one value in the routing information of the packet (e.g., in the header) at each hop along the route from the source to the destination). For example, in an embodiment where the address of the destination tile is specified by a relative position or distance from the current tile, each message router can route the message to a neighboring tile within the network that reduces the distance (e.g., in one of the two dimensions) and updates the packet data accordingly.
[0117] Thus, in operation 1504 (during its first execution), the router of the source tile (in the first iteration) determines the next tile along the path to the destination tile and routes the message within the source tile to the bidirectional photon channel connecting the source tile to the next tile. At the interface between the source tile and the bidirectional photon channel, at operation 1506, the message is applied to the optical signal, for example by operating the modulator driver of the source tile so that the associated modulator of the photon channel modulates the optical signal according to the message. At operation 1508, the modulated optical signal carrying the message is transmitted to the next tile. At operation 1510, the optical signal is converted back to an electronic signal at the interface between the photon channel and the next tile, for example, by measuring the optical signal using a photodetector at the end of the photon channel. In some embodiments, upon reaching the EIC of the next tile, the routing information of the packet is modified at operation 1511 to reflect that the packet has moved one step toward the destination in one of the multiple dimensions. Thereafter, the message router of the next tile has a packet with routing information whose fields have been modified to indicate the current number of steps remaining in a number of different dimensions towards the destination tile.
[0118] At operation 1512, it is determined whether the tile that received the message at operation 1510 is a destination tile or an intermediate tile. If the tile is an intermediate tile (e.g., in one embodiment, the routing information field in the header of the packet is non-zero), the method 1500 loops back to operation 1504, in which the intermediate tile (during the second and any subsequent executions) determines the next tile along the path to the destination. The message is then imparted onto an optical signal (at operation 1506) for transmission to the next tile (at operation 1508), and after conversion of the optical signal, the message is extracted from the resulting electronic signal (at operation 1510). The method 1500 traverses 1504-1512 until it reaches the destination tile. Once the destination tile's router determines that the message is at its destination, at operation 1514 (e.g., if both fields in the packet with routing information are zero), the message (or its payload, such as, for example, an activation streamed from a source tile) is routed to a local port within the destination tile, e.g., for transmission to a processing engine or memory of the destination tile.
[0119] Fig.16 16 is a flow chart illustrating an example method 1600 for implementing an ANN on an ML processor according to some embodiments. The ANN includes a plurality of neural nodes having associated predetermined weights and predetermined activation flows between the nodes. The ML processor (e.g., Figure 1A and Figure 1BThe SIP 100 implementation of the invention includes an electro-optical network including a plurality of processing elements (e.g., 104) connected via bidirectional photonic channels, each processing element including one or more processing blocks, a memory communicatively coupled to the one or more processing blocks, and a message router including two or more photonic channel interfaces. The method includes, at operation 1602, receiving input data in a machine-readable form to the electro-optical network of an ML processor, the input data including a representation of the ANN, the representation including weights. A connection to the ML processor may be established, for example, via PCIE, an ARM core, a general purpose processor, or other system of the ML processor that is electronically or optically coupled to the electro-optical network. Operation 1602 may include, for example, providing data to a compiler via TensorFlow, which converts the data from a human-readable form to a machine-readable form.
[0120] At operation 1604, based on the allocation between the neural nodes and the processing elements, the weights of the ANN are loaded into the processing elements of the electro-optical network. The allocation can be determined at compile time (e.g., before connecting to the electro-optical network at operation 1602). In another embodiment, the weights of the ANN are loaded into the local memory of the processing element of the electro-optical network by a scheduler that operates after compilation but before executing the reasoning or model. In some embodiments, each neural node is assigned to one of the processing elements, wherein each processing element can implement one or more neural nodes. In other embodiments, one or more neural nodes are each assigned to multiple processing elements that jointly implement the node. The weight associated with each neural node is loaded into the memory of one or more processing elements that are assigned to implement the neural node, for example, after the code is compiled but before the hardware is executed.
[0121] At operation 1606, the processing elements (and more specifically, for example, their message routers) are configured based on the assignment of nodes to the processing elements in conjunction with the predetermined activation flows between the nodes within the ANN. For example, the configuration may involve providing the router with path setting packet data in electronic form or via messages transmitted in a photonic manner. As a result of the configuration, the output activations calculated at each processing element will be routed to the corresponding processing elements (one or more) of the neural nodes (one or more) in the ANN, which receive these activations as inputs. The configuration is such that the activations between the neural nodes implemented by different processing elements in the processing elements are optically transmitted between the processing elements via a bidirectional photon channel, while the activations between the neural nodes implemented by the shared processing elements are electronically transmitted within the processing elements. At operation 1608, the ML processor is operated to implement the nodes of the ANN, for example, by executing code and performing calculations on the associated weights and input activations in the processing engine, and streaming the activations between the processing elements according to the routing decisions made by the configured routers.
[0122] The following numbered examples are illustrative embodiments and are not intended to be limiting.
[0123] 1. A circuit package, comprising: an electronic integrated circuit, comprising a plurality of processing elements, each processing element comprising a message router, the message router comprising a photon channel interface; and a plurality of bidirectional photon channels, which connect the processing elements to an electro-photonic network, each bidirectional photon channel being coupled at a first end to the photon channel interface of the message router of a first processing element among the processing elements and at a second end to the photon channel interface of the message router of a second processing element among the processing elements, and being configured to optically transmit packets between the message routers of the first processing element and the second processing element.
[0124] 2. The circuit package of Example 1, wherein each bidirectional photonic channel comprises one or more unidirectional photonic links in two directions.
[0125] 3. The circuit package of Example 2, wherein each bidirectional photonic channel comprises a plurality of unidirectional photonic links in two directions, and wherein the plurality of unidirectional photonic links in each direction are combined together.
[0126] 4. A circuit package according to Example 2 or Example 3, wherein each unidirectional photonic link includes an optical modulator at an input end, a photodetector at an output end, and an optical waveguide connecting the optical modulator to the photodetector, and wherein each photonic channel interface includes a modulator driver for each unidirectional photonic link connected to it at the input end, and a transimpedance amplifier for each unidirectional photonic link connected to it at the output end.
[0127] 5. The circuit package of Example 4, wherein the optical modulator is an electro-absorption modulator (EAM).
[0128] 6. The circuit package of Example 4, wherein the optical modulator comprises at least one of a ring modulator, a Mach-Zehnder modulator, and a quantum confined Stark effect (QCSE) electro-absorption modulator.
[0129] 7. The circuit package of any of Examples 4-6, wherein the electronic circuit is implemented in an electronic circuit layer, and the plurality of bidirectional photonic channels are implemented in a photonic circuit layer.
[0130] 8. The circuit package of example 7, wherein the electronic circuit layer is implemented in a first semiconductor chip, and the photonic circuit layer is implemented in a second semiconductor chip.
[0131] 9. The circuit package of Example 7 or Example 8, wherein each modulator driver is placed directly above or below the optical modulator associated therewith, and each transimpedance amplifier is placed directly above or below the photodetector associated therewith.
[0132] 10. The circuit package of example 8 wherein the distance between the optical modulator and its associated modulator driver and between the photodetector and its associated transimpedance amplifier is less than 200 μm.
[0133] 11. The circuit package according to any one of Examples 8-10 further includes an optical input waveguide in the photonic circuit layer of the semiconductor chip, which supplies an optical carrier signal provided by one or more laser sources to the optical modulator of the photonic link, and the optical carrier signal is modulated by the optical modulator according to a drive signal applied by an associated modulator driver to generate a modulated optical signal carrying the group.
[0134] 12. The circuit package of example 11, further comprising the one or more laser sources integrated into the circuit package.
[0135] 13. The circuit package of example 12, wherein the one or more laser sources are integrated with the optical input waveguides in one or more photonic integrated circuits (PICs) in the photonic circuit layer.
[0136] 14. The circuit package of Example 12, wherein the one or more laser sources are implemented in an interposer co-packaged with the semiconductor chip and are edge-coupled to one or more PICs in the photonic circuit layer including the optical input waveguide.
[0137] 15. The circuit enclosure of example 11, wherein the one or more laser sources are implemented external to the circuit enclosure and optically coupled to one or more PICs in the circuit enclosure via one or more optical fibers.
[0138] 16. The line enclosure of example 15, wherein the one or more optical fibers are coupled to the PICs via one or more fiber attachment units (FAUs) located above grating couplers implemented in the one or more PICs.
[0139] 17. The circuit enclosure of example 15, wherein the one or more optical fibers are edge coupled to the one or more PICs.
[0140] 18. The circuit package according to any one of Examples 15-17 further includes one or more optical splitters in the photonic circuit layer of the semiconductor chip, each optical splitter splitting an input optical carrier signal received from an associated laser source among the one or more laser sources between multiple input optical waveguides.
[0141] 19. The circuit package of any of Examples 1-18, wherein the electronic integrated circuit is an application specific integrated circuit (ASIC).
[0142] 20. The circuit package of example 19, wherein the ASIC includes digital and mixed signal electronic circuit systems.
[0143] 21. The circuit package of example 19, wherein each processing element further comprises a dot product engine that implements a combined tree of multipliers and adders for computing dot products between vectors.
[0144] 22. The circuit package of example 21, wherein each processing element further comprises a tensor engine for obtaining and processing vectors computed as outputs of the dot product engine.
[0145] 23. The circuit package of Example 21 or Example 22, wherein each processing element further comprises a memory system.
[0146] 24. The circuit package according to any one of Examples 19-23, further comprising at least one of a Peripheral Component Interconnect Express (PCIE) interface, an Advanced RISC Machine Core, an image processor, or an external memory controller.
[0147] 25. The circuit package of any of Examples 19-24, further comprising a high bandwidth memory (HBM), the ASIC further comprising a RAM controller communicatively coupling the processing element to the HBM.
[0148] 26. A circuit package according to any of Examples 1-25, wherein the message router of the processing element is configured to route packets, the packets having routing information in a portion of the packets, wherein a path through the electro-optical network is determined by using the routing information.
[0149] 27. A circuit package according to Example 26, wherein the message router of each processing element is configured to determine the destination of the packet based on the routing information in the packet received on one of its photon channel interfaces, and if the destination is the local memory or processing engine within the processing element, route the packet to a local port, or if the destination is not the local memory or processing engine within the processing element, route the packet to another photon channel interface.
[0150] 28. A circuit package according to Example 26 or Example 27, wherein the processing elements are arranged in a regular grid, and wherein the routing information in the group specifies the relative position of the processing element that is the destination of the group along the grid in a first dimension and a second dimension.
[0151] 29. A circuit package according to Example 28, wherein each bidirectional photon channel connects a pair of adjacent processing elements in the grid in one of the first dimension or the second dimension, and wherein the message router of each processing element is configured to decrement the value in the routing information associated with the relative position in the first dimension when the packet is routed to the adjacent processing element along the first dimension, and to decrement the value in the routing information associated with the relative position in the second dimension when the packet is routed to the adjacent processing element along the second dimension.
[0152] 30. A circuit package according to any of Examples 26-29, wherein the plurality of bidirectional photonic channels are configured to connect the processing elements to an electro-photonic network having a grid topology defining at least two dimensions, wherein the routing information in the packet specifies coordinates of the processing element in the at least two dimensions relative to the destination of the packet, and wherein the message router of each processing element modifies the routing information in the packet by decrementing a value associated with one of the at least two dimensions when routing the packet to a topologically adjacent processing element along one of the at least two dimensions.
[0153] 31. The circuit package of any of Examples 1-30, wherein the electronic circuit is configured as an arrangement of tiles, each tile corresponding to one of the processing elements.
[0154] 32. The circuit enclosure of example 31 wherein the tiles are rectangular tiles arranged in a rectangular grid.
[0155] 33. The circuit package of example 32, wherein the plurality of bidirectional photonic channels are configured to directly connect each pair of immediately adjacent tiles in the rectangular grid to produce a quadrilateral mesh topology of the electro-photonic network.
[0156] 34. The circuit package of Example 33, wherein the plurality of bidirectional photonic channels are configured to further directly connect one or more pairs of non-adjacent tiles in the rectangular grid.
[0157] 35. The circuit package of any of Examples 33-35, further comprising a bidirectional photonic channel configured to directly connect pairs of tiles on opposite edges of the rectangular grid to produce a wrapped torus mesh topology of the electro-photonic network.
[0158] 36. The circuit package of any of Examples 1-35, wherein the bidirectional photonic channel is configured to connect the processing element to an electro-photonic network having a cubic grid topology.
[0159] 37. A method comprising: generating, at a first electronic processing element in an electro-optical network, an electronic packet having a header and a payload, a portion of the payload or the header comprising routing information associated with a destination electronic processing element in the electro-optical network; determining, based on the routing information, a path from the first electronic processing element to the destination electronic processing element via one or more intermediate electronic processing elements in the electro-optical network, wherein the first processing element, the destination processing element, and the intermediate processing elements are connected by associated photonic channels; routing, at the first processing element, the packet to a destination electronic processing element via one of the photonic channels; The subchannel is connected to a port of a first intermediate processing element along the path and transforms the packet from an electronic signal to an optical signal for transmission to the first intermediate processing element; at each of the one or more intermediate processing elements, the optical signal is transformed back to an electronic signal, the electronic signal is routed to a port connected to the intermediate processing element and the next processing element along the path in the destination processing element via one of the photon channels, and the electronic signal is transformed to an optical signal for transmission to the next processing element along the path; and at the destination processing element, the optical signal is transformed back to an electronic signal.
[0160] 38. A method according to Example 37, wherein converting each electronic signal into an optical signal includes operating a modulator driver of a corresponding electronic processing element so that an associated modulator of the photon channel modulates the optical signal according to the grouping, and wherein converting the optical signal back into an electronic signal includes measuring the optical signal by a photodetector of the photon channel.
[0161] 39. A method according to Example 37 or Example 38, wherein each of the first electronic processing element, the intermediate electronic processing element, and the destination electronic processing element includes a router, and wherein the path through the electro-optical network is jointly determined by the routers of the first electronic processing element, the intermediate electronic processing element, and the destination electronic processing element.
[0162] 40. A method according to Example 39, wherein the routing information specifies the destination according to a relative position from the destination electronic processing element in a first dimension and a second dimension, and the method further includes modifying the routing information in the packet in response to transmitting the packet from either the first electronic processing element or the intermediate electronic processing element to a next electronic processing element along the path.
[0163] 41. A method according to Example 39 or Example 40, wherein the routers of the first electronic processing element, the intermediate electronic processing element, and the destination electronic processing element each include a local port and a port associated with a plurality of photonic channels that connect the corresponding electronic processing element to a plurality of other electronic processing elements within the electro-optical network, wherein the routers of the first electronic processing element and the intermediate electronic processing element each direct the packet to the port associated with the photonic channel connected to the next electronic processing element along the path, and wherein the router of the destination electronic processing element directs the packet to its local port.
[0164] 42. A system comprising: a first circuit package and a second circuit package, each circuit package comprising: an electronic integrated circuit EIC, which includes multiple processing elements and multiple message routers capable of transmitting and receiving messages, and multiple intra-chip bidirectional photonic channels, which connect the message routers to an intra-chip electro-photonic network capable of operating the messages; and multiple inter-chip bidirectional photonic channels, which connect the message routers of the first circuit package and the second circuit package to an inter-chip electro-photonic network, and the inter-chip electro-photonic network is capable of transmitting the message from the EIC in the first circuit package to the EIC in the second circuit package.
[0165] 43. The system of example 42, wherein each circuit package includes an electronic circuit layer in which the EIC is implemented and a photonic circuit layer in which the intra-chip bidirectional photonic channel is implemented.
[0166] 44. The system of example 43, wherein each circuit package comprises a first semiconductor chip implementing the electronic circuit layer and a second semiconductor chip implementing the photonic circuit layer.
[0167] 45. The system of any of Examples 42-44, wherein the plurality of on-chip bidirectional photonic channels of each of the circuit packages are implemented using optical waveguides in a photonic integrated circuit (PIC) of the circuit package.
[0168] 46. A system according to any of Examples 42-45, wherein the intra-chip bidirectional photonic channels each include one or more photonic links in each direction, each photonic link in the PIC including an optical modulator at the input end of the photonic link, a photodetector at the output end of the photonic link, and an optical waveguide connecting the optical modulator to the photodetector.
[0169] 47. A system according to Example 46, wherein the message router includes photonic line interfaces, each photonic line interface is used to couple with one of the bidirectional photonic channels, each photonic line interface includes a modulator driver for each photonic link it couples at the input end of the photonic link, and a transimpedance amplifier for each photonic link it couples at the output end of the photonic link.
[0170] 48. A system according to Example 46 or Example 47, wherein the multiple inter-chip bidirectional photonic channels are implemented using optical fiber.
[0171] 49. A system according to Example 48, wherein the first circuit package and the second circuit package each include one or more optical coupling structures for coupling the electro-optical network of the first circuit package to the electro-optical network of the second circuit package via the optical fiber.
[0172] 50. The system of Example 49, wherein the optical coupler comprises an edge coupler in the PIC.
[0173] 51. A system according to Example 49 or Example 50, wherein the optical coupling structure includes a fiber attachment unit FAU located above the grating coupler in the PIC.
[0174] 52. A system according to any of Examples 48-51, wherein the inter-chip bidirectional photonic channel is configured to transmit multiplexed optical signals.
[0175] 53. A system according to Example 52, wherein the inter-chip bidirectional photon channels each include a directional photon channel in each direction, and each directional photon channel between a first PIC of a first circuit package and a second PIC of a second circuit package includes: in the first PIC, a plurality of optical modulators for creating modulated optical signals at a plurality of corresponding wavelengths, a multiplexer for combining the modulated optical signals at the plurality of wavelengths into a single multiplexed optical signal, and an optical coupling structure for coupling the multiplexed optical signal into the optical fiber; and in the second PIC, an optical coupling structure for coupling the multiplexed optical signal from the optical fiber into the second PIC, a demultiplexer for demultiplexing the multiplexed optical signal into modulated optical signals at the plurality of wavelengths, and a plurality of corresponding photodetectors for measuring the modulated optical signals at the plurality of wavelengths.
[0176] 54. The system according to any one of Examples 42-53 further includes: at least one single-wavelength light source for supplying an optical carrier signal to the modulator of the bidirectional photonic channel within the chip; and at least one multi-wavelength light source for supplying optical carrier signals at multiple wavelengths to the modulator group of the bidirectional photonic channel between the chips.
[0177] 55. A system according to Example 54, wherein the at least one single wavelength light source and the at least one multi-wavelength light source include light sources integrated into the circuit package.
[0178] 56. A system according to Example 54, wherein the at least one single wavelength light source and the at least one multi-wavelength light source include light sources disposed outside the circuit enclosure and optically coupled to one or more PICs in the circuit enclosure via one or more optical fibers.
[0179] 57. The system of any of Examples 42-56, further comprising a plurality of additional circuit packages having an associated EIC with a plurality of associated processing elements, the first circuit package, the second circuit package, and the additional circuit packages being arranged in a quadrilateral grid.
[0180] 58. A system according to Example 57, wherein each of the intra-chip bidirectional photonic channels and the inter-chip bidirectional photonic channels connects a pair of adjacent processing elements in the quadrilateral grid to produce a quadrilateral mesh topology of the inter-chip electro-photonic network.
[0181] 59. A system according to Example 58, wherein the multiple inter-chip bidirectional photonic channels are further configured to directly connect pairs of processing elements on opposite edges of the quadrilateral grid to form an electro-photonic network having a scalable wrapped torus mesh topology.
[0182] 60. A system according to any of Examples 42-59, wherein the message router is configured to route the message through the inter-chip electro-photonic network and the intra-chip electro-photonic network, wherein when the message is sent from a first processing element to a second processing element that is not directly connected to the first processing element via a bidirectional photonic channel, the electrons are routed to the second processing element along a path through one or more third processing elements, wherein the message is converted from the optical domain to the electronic domain at each of the third processing elements and converted back to the optical domain at the message router associated with the corresponding third processing element.
[0183] 61. A system according to example 60, wherein the message router is configured to determine a path through the electro-optical network for each message based in part on a portion of the message associated with routing information.
[0184] 62. A processing device implementing an artificial neural network (ANN), the ANN comprising a plurality of neural nodes having associated weight sets, the processing device comprising: an electro-photonic network comprising a plurality of electronic processing elements connected via bidirectional photonic channels, each processing element comprising one or more processing engines, a memory communicatively coupled to the one or more processing engines, and a message router engaged with two or more of the bidirectional photonic channels and communicatively coupled to the memory, wherein: each neural node of the ANN is implemented by one or more of the processing elements; the weight set associated with each neural node is stored in the memory of the one or more processing elements implementing the neural node; the message router is configured to stream a plurality of activations between the processing elements implementing the corresponding neural nodes during execution of the ANN based on a predetermined activation flow between the neural nodes; and the one or more processing engines within each processing element are configured to, for each neural node implemented by the processing element, perform calculations on the associated weight set stored in the memory of the processing element and the input activation to the node streamed to the processing element to generate an output activation of the node during execution of the ANN.
[0185] 63. A processing device according to Example 62, wherein each processing element implements one or more of the neural nodes.
[0186] 64. A processing device according to Example 62 or Example 63, wherein at least one of the neural nodes is implemented jointly by a plurality of the processing elements.
[0187] 65. A processing device according to any of Examples 62-64, wherein activations are exchanged between neural nodes implemented by the same processing element via the memory of the processing element, and wherein activations are exchanged between neural nodes implemented by different processing elements via associated bidirectional photon channels in the bidirectional photon channels.
[0188] 66. The processing device of any of Examples 62-65, wherein the one or more processing engines include a dot product engine that implements a combinatorial tree of multipliers and adders configured to perform a multiply-accumulate MAC operation.
[0189] 67. A processing device according to Example 66, wherein the one or more processing engines further include a tensor engine configured to calculate non-linear activations.
[0190] 68. A processing device according to any one of Examples 62-67, wherein the multiple electronic processing elements are implemented on one or more electronic integrated circuits EICs, each EIC includes a plurality of the electronic processing elements, and wherein the bidirectional photonic channels connecting the electronic processing elements to each EIC are implemented as optical waveguides in an associated photonic integrated circuit PIC stacked with the EIC.
[0191] 69. The processing device of example 68, wherein each EIC and its associated PIC are packaged together as a system-in-package (SIP).
[0192] 70. The processing device of Example 68 or Example 69, wherein the plurality of electronic processing elements are implemented on a plurality of EICs, and wherein the bidirectional photonic channels connecting the electronic processing elements on different ones of the plurality of EICs are implemented using optical fibers.
[0193] 71. A processing device according to any of Examples 68-70, wherein the plurality of electronic processing elements are arranged in a quadrilateral grid, and wherein the bidirectional photonic channels are collectively configured to connect adjacent pairs of processing elements in the grid to produce a quadrilateral mesh topology of the electro-photonic network.
[0194] 72. A method of implementing an artificial neural network (ANN), the ANN comprising a plurality of neural nodes having associated predetermined weights, the method comprising: connecting to an electro-photonic network comprising a plurality of processing elements connected via bidirectional photonic channels, each processing element comprising one or more processing engines, a memory communicatively coupled to the one or more processing engines, and a message router comprising two or more photonic channel interfaces and a communicatively coupled to the memory; prior to run time, loading the predetermined weights associated with each neural node into one or more neural nodes assigned to implement the neural node based on an assignment between the neural nodes and the processing elements the memory of a plurality of processing elements; at runtime, using the message router to route the activations between the neural nodes implemented by the processing elements based on the allocation and a predetermined activation flow between the neural nodes within the ANN, wherein the activations routed between nodes implemented by different source and destination elements in the processing elements are transformed from electrical form to optical form for transmission via the bidirectional photonic channel at the source element and any intermediate processing element along a path from the source element to the destination element, and transformed from optical form to electrical form at the destination element and any intermediate processing element along the path.
[0195] 73. A method according to example 72, wherein the weights are loaded from a memory external to the electro-optical network into the memory of the processing element via a PCIE interface.
[0196] 74. A method according to example 72 or 73, wherein activations between neural nodes implemented by a shared processing element are transmitted electronically within the processing element.
[0197] 75. A method according to any of Examples 72-74, wherein the allocation between the neural nodes and the processing elements is determined at compile time.
[0198] 76. The method of any one of Examples 72-75, further comprising operating the electro-optical network to: stream activations between the processing elements based on routing decisions made by configured routers; and for each node, perform calculations on the associated weights and input activations streamed to the node using the one or more processing engines of the one or more processing elements assigned to implement the node.
[0199] 77. The method of any one of Examples 72-76, further comprising selectively activating a photonic link within the bidirectional optical channel based on the activation flow to reduce power consumption.
[0200] 78. A method according to Example 77, wherein each bidirectional photonic channel includes multiple photonic links, and wherein one of the links in each channel remains active to serve as a master link.
[0201] 79. A method according to any of Examples 72-78, wherein each bidirectional photonic channel includes multiple photonic links in each direction, and the method further includes establishing logical channels on at least some of the bidirectional photonic channels during initialization of the electro-photonic network, wherein establishing logical channels on the bidirectional photonic channels includes combining the photonic links in each direction within the bidirectional photonic channel together to form a combined group of links.
[0202] 80. The method of Example 79, wherein non-functional photonic links are omitted from the combined group.
[0203] 81. A method according to Example 79 or Example 80, wherein the number of photonic links in the combined group is determined based on bandwidth requirements associated with implementing the ANN.
[0204] 82. A hardware circuit comprising: a combinatorial tree having a plurality of rows and a plurality of columns, the combinatorial tree using a plurality of hardware units to generate one or more dot product values based on an input value received at a first column of the combinatorial tree; a first memory device for storing the input value; a sequencer configured to obtain the input value from the first memory device and to reassemble the input value in a switch structure; a clock configured to control the timing at which a portion of the reassembled input value is provided from the switch structure to the first column of the combinatorial tree; and a plurality of hardware units, each of which includes an accumulator configured to accumulate the one or more dot product values, wherein during one or more subsequent time periods defined by the clock, an additional portion of the reassembled input value is provided to the accumulator of the hardware unit associated with the first column, and wherein during one or more additional time periods controlled by the clock, a value associated with each of the accumulators of the hardware units in the first column is written to the first memory device.
[0205] 83. A hardware circuit according to Example 82, wherein the hardware unit includes at least one fused multiply-accumulate (FMA) unit.
[0206] 84. A hardware circuit according to any one of Examples 82-84, wherein the combination tree includes an array of dot product units configured to receive the input value and generate the one or more dot product values based on the input value received by the first column, and a single dot product unit of the array includes two or more of the hardware units.
[0207] 85. The hardware circuit according to Example 84 further includes a control circuit, the dot product unit array includes a data path, and the control circuit is configured to use the data path to interchange at least two weight values between two adjacent columns of the array, or to propagate weight values from a first row of the array to a second row of the array.
[0208] 86. The hardware circuit of any of Examples 82-85, wherein the sequencer comprises a switch structure that operably couples the combinational tree to the first memory device.
[0209] 87. A hardware circuit according to any one of Examples 82-86, wherein the sequencer determines a sequence of input vectors provided to the first column based on a parameter set, the parameter set including at least one of a stride value, a dilation value, or a kernel size value.
[0210] 88. A hardware circuit according to any one of Examples 82-87, wherein the sequencer is configured to generate individual input vectors of the input vector sequence by reading multiple weight values and multiple operand values from the first memory device during one or more clock cycles, the individual input vectors including the multiple weight values and the multiple operand values.
[0211] 89. A hardware circuit according to Example 88, wherein the first memory device includes a memory group, and wherein the multiple weight values and the multiple operand values read by the sequencer during the one or more clock cycles are selected for the reading, so that: the multiple weight values and the multiple operand values are arranged by the sequencer in the switching structure after being received from the memory group, and each of the weight values and the operand values is read by the sequencer in a single time.
[0212] 90. A hardware circuit according to Example 88, wherein the first memory device includes a memory group, and wherein the multiple weight values and the multiple operand values read by the sequencer during one or more clock cycles are selected for the reading, so that: the multiple weight values and the multiple operand values are arranged by the sequencer in the switching structure after being received from the memory group, and each of the weight values and the operand values is read by the sequencer in a single time; and data is read from each of the memory groups, wherein the number of rows of the memory groups is substantially equal to the number of rows in the first column of the combination tree.
[0213] 91. The hardware circuit of example 88, wherein each successive read of the first memory device by the sequencer causes a new weight value and a new operand value to be read from the first memory device that were not read by the sequencer during a previous clock cycle.
[0214] 92. A hardware circuit according to any of Examples 82-91, wherein a single hardware unit is operably coupled to at least two adjacent hardware units so that an output value generated by the single hardware unit is shared as an input value of each of the at least two adjacent hardware units.
[0215] 93. A hardware circuit according to any of Examples 82-92, wherein the accumulator includes a plurality of accumulators configured to accumulate partial dot product values generated by a single hardware unit as values flow through the single hardware unit.
[0216] 94. A hardware circuit according to any of Examples 82-93, wherein the combinational tree includes registers associated with the hardware unit, and the registers are configured to maintain clock alignment of values flowing through the hardware unit.
[0217] 95. A hardware circuit according to any one of Examples 82-94, comprising logic configured to generate a final dot product value set for the input values after the input vector sequence has been fully input into the combination tree, the final dot product value set being composed of values stored in the accumulator of the hardware unit of the combination tree.
[0218] 96. The hardware circuit of example 95, wherein the logic is configured to store the final set of dot product values to the first memory device.
[0219] 97. A hardware circuit according to any of Examples 82-96, wherein the input value includes at least one weight value and at least one operand value.
[0220] 98. The hardware circuit of any one of Examples 82-97, wherein the hardware circuit is part of a photonic integrated circuit (PIC).
[0221] 99. A method comprising: reading, by a sequencer, a first plurality of weight values and a first plurality of operand values from a memory device set operably coupled to the sequencer; receiving, at a first column of a combinatorial tree, a plurality of first input vectors from the sequencer, the first plurality of input vectors comprising the first plurality of weight values and the first plurality of operand values; processing the first plurality of input vectors by a math unit at the first column of the combinatorial tree, the math unit being associated with each of the input vectors, each of the math units having at least one accumulator, the combinatorial tree being configured to generate a dot product value in the accumulator of each of the math units based on the value received by the math unit at the first column of the combinatorial tree; reading, by the sequencer, a second plurality of weight values from the memory device set; receiving, by the sequencer, a plurality of first input vectors at a first column of a combinatorial tree, the first plurality of input vectors comprising the first plurality of weight values and the first plurality of operand values; processing the first plurality of input vectors by a math unit at the first column of the combinatorial tree, the math unit being associated with each of the input vectors, each of the math units having at least one accumulator, the combinatorial tree being configured to generate a dot product value in the accumulator of each of the math units based on the value received by the math unit at the first column of the combinatorial tree; a plurality of weight values and a second plurality of operand values; receiving a second plurality of input vectors from the sequencer at the first column of the combinatorial tree, the second plurality of input vectors comprising the second plurality of weight values and the second plurality of operand values; processing the second plurality of input vectors by the math unit at the first column of the combinatorial tree, the first plurality of input vectors and the second plurality of input vectors being part of a sequence of input vectors to be received by the math unit in the first column of the combinatorial tree; and providing a final set of dot product values by retrieving a value from each accumulator in each of the math units residing in each row of the first column of the combinatorial tree after the sequence of input vectors has all been received by the first column of the combinatorial tree and processed by the associated math unit.
[0222] 100. A method according to Example 99, wherein the at least one accumulator includes multiple accumulators in each of the math units, and the multiple accumulators are configured to accumulate partial dot product values generated by the individual math units as values flow through the individual math units.
[0223] 101. A hardware circuit comprising: an array of dot product cells organized as a combinatorial tree, the combinatorial tree generating dot product values based on weight values and activation values received as input by the dot product cell array, the dot product cell array comprising 64 rows by 32 columns of dot product cells, each dot product cell being configured to receive two 8-bit values as input and having an accumulator; and a switch structure operably coupling the array to a set of memory devices, the set of memory devices storing a plurality of weight values and a plurality of activation values, the switch structure being configured to provide 64 input vectors as input to a first column of the array during a first clock sequence comprising one or more clock cycles, wherein one or more additional clock sequences cause the switch structure to provide multiple additional sets of 64 input vectors to the first column of the array, and wherein when each of the accumulators in the first column of the array has 32 bits, each value from each accumulator is written back to the set of memory devices.
[0224] 102. The hardware circuit of example 101, wherein the dot product unit comprises a plurality of FMA units.
[0225] 103. The hardware circuit of Example 101 or Example 102, wherein the dot product unit is configured to receive a 32-byte input.
[0226] 104. The hardware circuit of any of Examples 101-103, wherein the set of memory devices comprises 64 groups of 32-byte wide memory devices.
[0227] 105. A circuit package, comprising: an application specific integrated circuit ASIC, comprising a plurality of processing elements, the plurality of processing elements comprising a photon channel interface; and a plurality of photon channels, which connect the plurality of processing elements via the photon channel interface to form at least a portion of an electro-photonic network, a sub-multiple photon channels of the plurality of photon channels connected to individual processing elements of the plurality of processing elements, the individual processing elements being configured to: select a master photon channel from the sub-multiple photon channels for the individual processing element; select a slave photon channel set from the sub-multiple photon channels for the individual processing element; extract a received clock signal from the master photon channel; generate a local clock signal based on the received clock signal, the individual processing element using the local clock A clock signal is used to perform one or more local operations on the individual processing element; and the distributed clock signal is transmitted to the set of the plurality of processing elements connected to the individual processing element through the set of slave photonic channels, the transmitting of the distributed clock signal includes using the photonic channel interface of the individual processing element to convert the distributed clock signal from an electrical form to an optical form, the transmitting of the distributed clock signal further includes converting the distributed clock signal from the optical form to the electrical form at the set of the plurality of processing elements, the distributed clock signal is suitable for transmission across the set of slave photonic channels and is suitable for use as an associated local clock signal by the set of the plurality of processing elements, the distributed clock signal is generated based on the received clock signal.
[0228] 106. A circuit package according to Example 105, wherein the individual processing element includes a jitter attenuation phase-locked loop, i.e., a jitter attenuation PLL, wherein the individual processing element uses the jitter attenuation PLL to generate a low-jitter clock signal based on the received clock signal, and wherein generating the local clock signal based on the received clock signal includes generating the local clock signal based on the low-jitter clock signal.
[0229] 107. The circuit package of example 105 or 106, wherein the individual processing element selects the primary photon channel during a boot time of the individual processing element.
[0230] 108. The circuit package of any of Examples 105-107, wherein the individual processing element selects the set of slave photon channels during a boot time of the individual processing element.
[0231] 109. A circuit package according to any of Examples 105-108, wherein the individual processing elements are configured to: perform a channel alignment operation on the main photon channel, and the channel alignment operation is configured to adjust one or more settings of the individual processing elements to compensate for clock skew.
[0232] 110. The circuit package of any of Examples 105-109, wherein the individual processing elements extract the received clock signal from the main photon channel by performing clock data recovery on an incoming bit stream received through the main photon channel.
[0233] 111. A circuit package according to any of Examples 105-110, wherein the individual processing elements transmit the distributed clock signal to the set of processing elements by transmitting one or more outgoing bit streams to the set of processing elements, and the one or more outgoing bit streams are generated based on the distributed clock signal.
[0234] 112. A circuit package according to any one of Examples 105-111, wherein the individual processing elements include a tensor processor and a DNN, wherein each of the tensor processor and the DNN is configured to operate based on the local clock signal, and wherein packets can be passed between the DNN and the tensor processor using the local clock signal.
[0235] 113. The circuit package of any of Examples 105-112, wherein the individual processing elements include a message router, the message router including the photonic channel interface, the message router configured to operate based on the local clock signal.
[0236] 114. A circuit package according to any of Examples 105-113, wherein the individual processing elements include hardware circuits for calculating one or more dot products between at least two vectors, and the hardware circuits are configured to operate based on the local clock signal.
[0237] 115. A circuit package according to any one of Examples 105-114, wherein the individual processing element is a first processing element, the sub-multiple photon channels are a first sub-multiple photon channels, the master photon channel is a first master photon channel, the set of slave photon channels is a first set of slave photon channels, the received clock signal is a first received clock signal, the local clock signal is a first local clock signal, and the distributed clock signal is a first distributed clock signal; and wherein a second sub-multiple photon channels among the multiple photon channels are connected to a second processing element, the second processing element is included in the set of processing elements connected to the first processing element, the second sub-multiple photon channels among the multiple photon channels include individual photon channels in the set of slave photon channels connecting the first processing element to the second processing element, and the second processing element is configured to: select a second master photon channel from the individual photon channels for the second processing element; select a second slave photon channel from the second sub-multiple photon channels for the second processing element a second set of slave photon channels; extracting a second received clock signal from the second master photon channel; generating a second local clock signal based on the second received clock signal, the second processing element using the second local clock signal to perform one or more local operations on the second processing element; and transmitting a second distributed clock signal to a second set of the plurality of processing elements connected to the second processing element through the second set of slave photon channels, transmitting the second distributed clock signal comprising converting the second distributed clock signal from the electrical form to the optical form using the photon channel interface of the individual processing element, transmitting the second distributed clock signal further comprising converting the second distributed clock signal from the optical form to the electrical form at the second set of the plurality of processing elements, the second distributed clock signal being suitable for transmission across the second set of slave photon channels and being suitable for use as an associated local clock signal by the second set of the plurality of processing elements, the second distributed clock signal being generated based on the second received clock signal.
[0238] 116. A circuit package according to Example 115, wherein the first processing element includes a first set of components, and the second processing element includes a second set of components, the first set of components including a first tensor processor, a first message router including a photon channel interface, or at least one of a first hardware circuit for calculating one or more first dot products between at least two vectors, the second set of components including a second tensor processor, a second message router including a photon channel interface, or at least one of a second hardware circuit for calculating one or more second dot products between at least two vectors, the first set of components is configured to operate based on the first local clock signal, and the second set of components is configured to operate based on the second local clock signal.
[0239] 117. A method comprising: selecting, by an individual processing element of a plurality of processing elements, a master photon channel from a plurality of photon channels, the master photon channel being selected for the individual processing element, the plurality of processing elements being part of an application specific integrated circuit ASIC of a circuit package, the plurality of processing elements being connected together through a plurality of photon channels to form at least a portion of an electro-optical network within the circuit package; selecting, by the individual processing element, a set of slave photon channels from the plurality of photon channels, the set of slave photon channels being selected for the individual processing element; extracting, by the individual processing element, a received clock signal from the master photon channel in a digital format; generating, by the individual processing element, a local clock signal based on the received clock signal in the digital format, the individual processing element using the local clock signal a clock signal to perform one or more local operations on the individual processing element using the digital portion of the ASIC; and transmitting the distributed clock signal by the individual processing element and through the set of slave photonic channels to the set of the plurality of processing elements connected to the individual processing element, the distributed clock signal being generated based on the received clock signal in the digital format, the transmitting comprising: converting the distributed clock signal from the digital format to an optical format; routing the distributed clock signal in the optical format across the set of slave photonic channels; receiving the distributed clock signal in the optical format at a photonic interface of the set of the plurality of processing elements; and converting the distributed clock signal to the digital format in the set of the plurality of processing elements using an associated photonic channel interface.
[0240] 118. A method according to Example 117, wherein the individual processing element includes a jitter attenuation phase-locked loop, i.e., a jitter attenuation PLL, wherein the individual processing element uses the jitter attenuation PLL to generate a low-jitter clock signal based on the received clock signal, and wherein generating the local clock signal based on the received clock signal includes generating the local clock signal based on the low-jitter clock signal.
[0241] 119. The method of Example 117 or Example 118, wherein selecting the primary photon channel is performed during a boot time of the individual processing element.
[0242] 120. The method of any of Examples 117-119, wherein the set of slave photon channels is selected during a boot time of the individual processing element.
[0243] 121. The method of any one of Examples 117-120, comprising: performing a channel alignment operation on the main photon channel by the individual processing element, the channel alignment operation being configured to adjust one or more settings of the individual processing element to compensate for clock skew.
[0244] 122. A method according to any of Examples 117-121, wherein extracting the received clock signal from the main photon channel includes performing clock data recovery on an incoming bit stream received through the main photon channel.
[0245] 123. A method according to any of Examples 117-122, wherein transmitting the distributed clock signal to the set of processing elements includes transmitting one or more outgoing bit streams to the set of processing elements, wherein the one or more outgoing bit streams are generated based on the distributed clock signal.
[0246] 124. A method according to any of Examples 117-123, wherein the individual processing element includes a set of components, the set of components including at least one of a tensor processor, a message router including a photon channel interface, or a hardware circuit for calculating a dot product between at least two vectors, and the set of components is configured to operate based on the local clock signal.
[0247] As can be seen from the above description, various techniques can be used to implement those concepts without departing from the scope of the concepts described in this application. In addition, although concepts have been described with specific reference to certain embodiments, it will be appreciated by those of ordinary skill in the art that changes can be made in form and detail without departing from the scope of those concepts. Therefore, the described embodiments are considered to be illustrative and not restrictive in all respects. It should also be understood that the application is not limited to the particular embodiments described herein, but many rearrangements, modifications and replacements are possible without departing from the scope of the present disclosure.
Claims
1. A circuit package, comprising: an electronic integrated circuit comprising a plurality of processing elements, each processing element comprising a message router, the message router comprising a photonic channel interface; as well as a plurality of bidirectional photonic channels connecting the processing elements into an electro-optical network, each bidirectional photonic channel being coupled at a first end to a photonic channel interface of the message router of a first one of the processing elements and at a second end to a photonic channel interface of the message router of a second one of the processing elements, and being configured to optically communicate packets between the message routers of the first and second processing elements, Each bidirectional photon channel includes one or more unidirectional photon links in two directions, and wherein each unidirectional photonic link comprises an optical modulator at an input end, a photodetector at an output end, and an optical waveguide connecting the optical modulator to the photodetector, and wherein each photonic channel interface comprises a modulator driver for each unidirectional photonic link it engages at the input end, and a transimpedance amplifier for each unidirectional photonic link it engages at the output end.
2. The circuit package of claim 1, wherein each bidirectional photonic channel comprises a plurality of unidirectional photonic links in two directions, and wherein the plurality of unidirectional photonic links in each direction are combined together.
3. The circuit package of claim 1, wherein the optical modulator is an electro-absorption modulator (EAM). 4 . The circuit package of claim 1 , wherein the optical modulator comprises at least one of a ring modulator, a Mach-Zehnder modulator, and a quantum confined Stark effect (QCSE) electro-absorption modulator.
5. The circuit package of claim 1, wherein the electronic integrated circuit is implemented in an electronic circuit layer, and the plurality of bidirectional photonic channels are implemented in a photonic circuit layer. 6 . The circuit package according to claim 5 , wherein the electronic circuit layer is implemented in a first semiconductor chip and the photonic circuit layer is implemented in a second semiconductor chip.
7. The circuit package of claim 5, wherein each modulator driver is placed directly above or below the optical modulator associated therewith, and each transimpedance amplifier is placed directly above or below the photodetector associated therewith.
8. The circuit package of claim 6, wherein the distance between the optical modulator and its associated modulator driver and between the photodetector and its associated transimpedance amplifier is less than 200 μm.
9. The circuit package of claim 6, further comprising an optical input waveguide in the photonic circuit layer of the semiconductor chip, which supplies an optical carrier signal provided by one or more laser sources to the optical modulator of the photonic link, the optical carrier signal being modulated by the optical modulator according to a drive signal applied by an associated modulator driver to generate a modulated optical signal carrying the packet.
10. The circuit package of claim 9, further comprising the one or more laser sources integrated into the circuit package.
11. The circuit package of claim 10, wherein the one or more laser sources are integrated with the optical input waveguides in one or more photonic integrated circuits (PICs) in the photonic circuit layer.
12. The circuit package of claim 10, wherein the one or more laser sources are implemented in an interposer co-packaged with the semiconductor chip and are edge-coupled to one or more PICs in the photonic circuit layer including the optical input waveguide.
13. The circuit enclosure of claim 9, wherein the one or more laser sources are implemented external to the circuit enclosure and optically coupled to one or more PICs in the circuit enclosure via one or more optical fibers.
14. The line enclosure of claim 13, wherein the one or more optical fibers are coupled to the PICs via one or more fiber attachment units (FAUs) located above grating couplers implemented in the one or more PICs.
15. The circuit enclosure of claim 13, wherein the one or more optical fibers are edge coupled to the one or more PICs.
16. The circuit package of claim 13, further comprising one or more optical splitters in the photonic circuit layer of the semiconductor chip, each optical splitter splitting an input optical carrier signal received from an associated laser source of the one or more laser sources between a plurality of input optical waveguides.
17. The circuit package of claim 1, wherein the electronic integrated circuit is an application specific integrated circuit (ASIC).
18. The circuit package of claim 17, wherein the ASIC comprises digital and mixed signal electronic circuit systems.
19. The circuit package of claim 17, wherein each processing element further comprises a dot product engine that implements a combined tree of multipliers and adders for computing dot products between vectors.
20. The circuit package of claim 19, wherein each processing element further comprises a tensor engine for obtaining and processing vectors computed as output of the dot product engine.
21. The circuit package of claim 19, wherein each processing element further comprises a memory system.
22. The circuit package of claim 17, further comprising at least one of a peripheral component interconnect express (PCIE) interface, an advanced RISC machine core, a graphics processor, or an external memory controller.
23. The circuit package of claim 17, further comprising a high bandwidth memory (HBM), the ASIC further comprising a RAM controller communicatively coupling the processing element to the HBM.
24. The circuit enclosure of claim 1, wherein the message router of the processing element is configured to route packets having routing information in a portion of the packets, wherein a path through the electro-optical network is determined by using the routing information.
25. The circuit enclosure of claim 24, wherein the message router of each processing element is configured to determine a destination of the packet based on the routing information in the packet received on one of its photon channel interfaces, and to route the packet to a local port if the destination is a local memory or a processing engine within the processing element, or to route the packet to another of the photon channel interfaces if the destination is not the local memory or the processing engine within the processing element.
26. The circuit package of claim 24, wherein the processing elements are arranged in a regular grid, and wherein the routing information in the packet specifies a relative position in a first dimension and a second dimension along the grid to a processing element that is a destination for the packet.
27. A circuit package according to claim 26, wherein each bidirectional photon channel connects a pair of adjacent processing elements in the grid in one of the first dimension or the second dimension, and wherein the message router of each processing element is configured to decrement the value in the routing information associated with the relative position in the first dimension when the packet is routed to the adjacent processing element along the first dimension, and to decrement the value in the routing information associated with the relative position in the second dimension when the packet is routed to the adjacent processing element along the second dimension.
28. A circuit package according to claim 24, wherein the plurality of bidirectional photonic channels are configured to connect the processing elements to an electro-optical network having a grid topology defining at least two dimensions, wherein the routing information in the packet specifies coordinates in the at least two dimensions relative to the processing element that is the destination of the packet, and wherein the message router of each processing element modifies the routing information in the packet by decrementing a value associated with one of the at least two dimensions when routing the packet to a topologically adjacent processing element along one of the at least two dimensions.
29. The circuit package of claim 1, wherein the electronic circuit is configured as an arrangement of tiles, each tile corresponding to one of the processing elements.
30. The circuit enclosure of claim 29, wherein the tiles are rectangular tiles arranged in a rectangular grid.
31. The circuit package of claim 30, wherein the plurality of bidirectional photonic channels are configured to directly connect each pair of immediately adjacent tiles in the rectangular grid to produce a quadrilateral mesh topology of the electro-photonic network.
32. The circuit package of claim 31, wherein the plurality of bidirectional photonic channels are configured to further directly connect one or more pairs of non-adjacent tiles in the rectangular grid.
33. The circuit package of claim 31, further comprising a bidirectional photonic channel configured to directly connect pairs of tiles on opposite edges of the rectangular grid to produce a wrapped torus mesh topology of the electro-photonic network.
34. The circuit package of claim 1, wherein the bidirectional photonic channel is configured to connect the processing element to an electro-photonic network having a cubic grid topology.
Citation Information
Cited By
Electrical bridge package with integrated off-bridge photonic channel interface
US12191257B2
Electro-photonic network for machine learning
US12353006B2
Via formed using a partial plug that stops before a substrate
US12568809B2
Coherent photonic computing architectures
US12615087B2
