Photonic system, computing system, and method of manufacturing a semiconductor wafer

Abstract

Described herein is a photonic communications platform that can overcome the memory bottleneck problem, thereby enabling scaling of memory capacity and bandwidth well beyond that possible with conventional computing systems. Some embodiments provide a photonic communications platform that involves the use of photonic modules. Each photonic module contains programmable photonic circuitry that places the module in optical communication with other modules based on the needs of a particular application. The architecture developed by the inventors relies on fabricating multiple photonic modules on a single wafer using a common photomask set (or at least one common photomask). The photonic modules within the multiple wafers can be coupled to the communications platform using optical or electronic means.

Description

Technical Field

[0001] This application relates to photonic System, computing system, and method of manufacturing a semiconductor wafer .

Background Art

[0002] A computer system includes a random access memory (RAM) for storing data and machine code. The RAM is typically a volatile memory such that the stored information is lost when power is removed. In recent implementations, the memory is in the form of an integrated circuit. Each integrated circuit includes a plurality of memory cells. To enable access to the stored data and machine code, the memory is arranged in electrical communication with a processor. Typically, these electrical communications are implemented as metal traces formed on a substrate on which the memory and the processor are provided.

Summary of the Invention

Means for Solving the Problems

[0003] Some embodiments relate to a photonic system including a plurality of photonic modules including at least a first photonic module and a second photonic module patterned according to at least one common photomask. The first photonic module and the second photonic module each include a first boundary line and a second boundary line, an optical distribution network, a first optical waveguide optically coupling the optical distribution network to a first neighboring photonic module of the plurality of photonic modules that is adjacent to the first boundary line, and a second optical waveguide optically coupling the optical distribution network to a second neighboring photonic module of the plurality of photonic modules that is adjacent to the second boundary line.

[0004] In some embodiments, the first boundary line and the second boundary line are in opposite positions relative to each other. In some embodiments, the first optical waveguide and the second optical waveguide are patterned according to at least one common photomask.

[0005] In some embodiments, the first photonic module and the second photonic module each further include an out-of-plane optical coupler optically coupled to an optical distribution network.

[0006] In some embodiments, the optical distribution network is configured to be selectively arranged to optically communicate a first neighboring photonic module with a second neighboring photonic module.

[0007] In some embodiments, the first photonic module and the second photonic module are each patterned according to a common set of photomasks, and at least one common photomask is part of the common set of photomasks.

[0008] In some embodiments, the first photonic module and the second photonic module are each a third boundary line and a fourth boundary line, where the first boundary line and the second boundary line are in opposite positions to each other, and the third boundary line and the fourth boundary line are in opposite positions to each other. A third optical waveguide that optically couples the optical distribution network to a third neighboring photonic module among the plurality of photonic modules, the third neighboring photonic module being adjacent to the third boundary line, and a fourth neighboring photonic module among the plurality of photonic modules, the fourth neighboring photonic module being adjacent to the fourth boundary line. And a fourth optical waveguide that optically couples the optical distribution network to the fourth neighboring photonic module.

[0009] In some embodiments, the optical distribution network is configured to be selectively arranged to optically communicate a first neighboring photonic module with a second neighboring photonic module or a third neighboring photonic module.

[0010] In some embodiments, the optical distribution network includes a plurality of optical switches. In some embodiments, the first photonic module and the second photonic module are adjacent to each other, and the second photonic module is the first neighboring photonic module of the first photonic module.

[0011] Some embodiments are methods for manufacturing a semiconductor wafer, including patterning at least some of a plurality of photonic modules on the semiconductor wafer using at least one common photomask, wherein patterning at least some of the plurality of photonic modules includes patterning the optical distribution network, patterning a first optical waveguide that optically couples the optical distribution network to a first neighboring photonic module of the plurality of photonic modules, which is a first neighboring photonic module adjacent to a first boundary line of the photonic module, and patterning a second optical waveguide that optically couples the optical distribution network to a second neighboring photonic module of the plurality of photonic modules, which is a second neighboring photonic module adjacent to a second boundary line of the photonic module.

[0012] In some embodiments, patterning at least some of the plurality of photonic modules includes patterning the first optical waveguide and the second optical waveguide using at least one common photomask.

[0013] In some embodiments, the method further includes obtaining a photonic substrate by dicing a semiconductor wafer to obtain a first photonic module among a plurality of photonic modules, a first neighboring photonic module adjacent to a first boundary line of the first photonic module, and a second neighboring photonic module adjacent to a second boundary line of the first photonic module.

[0014] In some embodiments, the first boundary line and the second boundary line of the first photonic module are in opposite positions to each other. In some embodiments, patterning at least some of the plurality of photonic modules respectively further includes patterning a third optical waveguide that optically couples an optical distribution network to a third neighboring photonic module among the plurality of photonic modules, the third neighboring photonic module being adjacent to a third boundary line of the photonic module, and patterning a fourth optical waveguide that optically couples an optical distribution network to a fourth neighboring photonic module among the plurality of photonic modules, the fourth neighboring photonic module being adjacent to a fourth boundary line of the photonic module. The first boundary line and the second boundary line are in opposite positions to each other, and the third boundary line and the fourth boundary line are in opposite positions to each other.

[0015] In some embodiments, patterning at least some of the plurality of photonic modules respectively further includes patterning the first photonic module using a first photolithography shot with respect to at least one common photomask, and patterning the second photonic module using a second photolithography shot with respect to at least one common photomask following the first photolithography shot.

[0016] Some embodiments relate to a computing system including at least a first photonic module and a second photonic module, a photonic substrate patterned with a plurality of photonic modules each patterned according to at least one common photomask and having the first photonic module optically coupled to the second photonic module, a first die communicating with the first photonic module, and a second die communicating with the second photonic module.

[0017] In some embodiments, the first die includes a processor and the second die includes memory. In some embodiments, the computing system further includes a laser die coupled to the photonic substrate.

[0018] In some embodiments, the first photonic module and the second photonic module each include a first boundary line and a second boundary line, an optical distribution network, a first optical waveguide optically coupling the optical distribution network to a first neighboring photonic module of the plurality of photonic modules that is adjacent to the first boundary line, and a second optical waveguide optically coupling the optical distribution network to a second neighboring photonic module of the plurality of photonic modules that is adjacent to the second boundary line.

[0019] In some embodiments, the first boundary line and the second boundary line are in opposite positions. In some embodiments, the first optical waveguide and the second optical waveguide are patterned according to at least one common photomask.

[0020] In some embodiments, the first photonic module and the second photonic module each include an out-of-plane optical coupler, the first die is optically coupled to the out-of-plane optical coupler of the first photonic module, and the second die is optically coupled to the out-of-plane optical coupler of the second photonic module.

[0021] In some embodiments, the first die is coupled to a first side of the photonic substrate, and the second die is coupled to a second side of the photonic substrate located at a position opposite to the first side.

[0022] In some embodiments, the computing system further includes a third die stacked on top of the first die. In some embodiments, the first photonic module and the second photonic module share a boundary line such that the first photonic module is adjacent to the second photonic module.

[0023] In some embodiments, the first die is attached above or below the first photonic module, and the second die is attached above or below the second photonic module.

[0024] In some embodiments, the first die communicates electronically with the first photonic module, and the second die communicates electronically with the second photonic module. Some embodiments relate to a multi-node computing system including at least a first computing system and a second computing system, each including a plurality of photonic modules, each of which is patterned with at least a first photonic module and a second photonic module patterned according to at least one common photomask, and the first photonic module is optically coupled to the second photonic module; a photonic substrate; a first die communicating with the first photonic module; a second die communicating with the second photonic module; and a fiber connecting the first computing system and the second computing system to each other.

[0025] In some embodiments, the first computing system and the second computing system each further include a fiber coupler, and the fiber optically couples the respective fiber couplers of the first computing system and the second computing system to each other.

[0026] In some embodiments, the first die includes a processor and the second die includes memory. In some embodiments, the first computing system and the second computing system each further include a laser die coupled to the photonic substrate.

[0027] In some embodiments, the first photonic module and the second photonic module of the photonic substrate each include a first boundary line and a second boundary line, an optical distribution network, a first optical waveguide that optically couples the optical distribution network to a first neighboring photonic module among the plurality of photonic modules, the first neighboring photonic module being adjacent to the first boundary line, and a second optical waveguide that optically couples the optical distribution network to a second neighboring photonic module among the plurality of photonic modules, the second neighboring photonic module being adjacent to the second boundary line.

[0028] In some embodiments, the first optical waveguide and the second optical waveguide are patterned according to at least one common photomask. In some embodiments, the first photonic module and the second photonic module each include an out-of-plane optical coupler, the first die is optically coupled to the out-of-plane optical coupler of the first photonic module, and the second die is optically coupled to the out-of-plane optical coupler of the second photonic module.

[0029] In some embodiments, the first die is coupled to a first side of the photonic substrate, and the second die is coupled to a second side of the photonic substrate located opposite the first side.

[0030] In some embodiments, the multi-node computing system further includes a third die stacked on top of the first die. In some embodiments, the first photonic module and the second photonic module share a boundary line such that the first photonic module is adjacent to the second photonic module.

[0031] In some embodiments, the first die is attached above or below the first photonic module, and the second die is attached above or below the second photonic module.

[0032] Some embodiments relate to a photonic communication platform including a photonic network including a plurality of optical switches formed on a semiconductor substrate, a plurality of dies communicating with the photonic network, and an electronic switching network including a plurality of transistors integrated with the plurality of optical switches, the electronic switching network configured to program the optical switches to form a first optical communication path coupling a first subset of the plurality of dies at a first time and to program the optical switches to form a second optical communication path coupling a second subset of the plurality of dies at a second time following the first time, the second optical communication path being distinct from the first communication path.

[0033] In some embodiments, the plurality of transistors are formed on the semiconductor substrate. In some embodiments, the semiconductor substrate is a first semiconductor substrate, the plurality of transistors are formed on a second semiconductor substrate, and the first semiconductor substrate and the second semiconductor substrate are 3D bonded.

[0034] In some embodiments, programming the optical switches to form the first optical communication path includes identifying an optical communication path coupling a first subset of the plurality of dies and programming the optical switches based on the identified optical communication path.

[0035] In some embodiments, identifying an optical communication path coupling a first subset of the plurality of dies includes monitoring use of the photonic network. In some embodiments, the electronic switching network is further configured to determine at least one characteristic of an optical signal on the first optical communication path, identify a coding system based on the at least one characteristic of the optical signal, and optically communicate with the photonic network on the first optical communication path based on the coding system.

[0036] In some embodiments, multiple dies communicate electronically with a photonic network. In some embodiments, the electronic switching network is further configured to optically communicate with the photonic network on a first optical communication path using wavelength division multiplexing.

[0037] Various aspects and embodiments of the present application will be described with reference to the following drawings. It should be understood that the drawings are not necessarily drawn to scale. Items that appear in multiple drawings are denoted by the same reference numerals in the drawings in which they appear.

Brief Description of the Drawings

[0038]

Figure 1

Figure 2A

Figure 2B

Figure 2C

Figure 2D

Figure 2E

Figure 3A

Figure 3B

Figure 3C

Figure 3D

Figure 3E

Figure 3F

Figure 3G

Figure 3H

Figure 3I

Figure 4

Figure 5A

Figure 5B

Figure 5C

Figure 5D

Figure 6A

Figure 6B

Figure 6C

Figure 7A

Figure 7B

Figure 7C

Figure 8A

Figure 8B

DETAILED DESCRIPTION OF THE INVENTION

[0039] I. Memory Bandwidth Scalability The inventors have recognized and evaluated that one of the major bottlenecks limiting the spread of data-intensive computing is that modern computers cannot scale memory capacity and bandwidth fast enough. The inventors have developed a photonic communication platform that enables scaling of memory capacity and bandwidth far beyond what is possible with conventional computers.

[0040] In data-intensive computing systems, such as systems designed to handle deep learning algorithms, a huge amount of data needs to be accessed, increasing the requirements for memory capacity. Furthermore, in most applications, data needs to be accessed in real time or near real time, increasing the requirements for memory bandwidth. Some conventional computing systems use a graphics processing unit (GPU) to improve memory access efficiency. Some GPUs can transfer data from memory at a bandwidth as high as 256 GB / second. Such memory bandwidth may be sufficient for most graphics-based applications, but it is far from sufficient for data-intensive applications such as deep neural networks and high-frequency trading.

[0041] Deep neural networks depend on a large number of parameters, such as weights and activation parameters. For example, a typical 50-layer network with 26 million weight parameters can compute up to 16 million activations in the forward pass. When using 32-bit floating-point values to store weight parameters and activation parameters, the total storage requirement is 168MB. Additionally, if the data is represented as a dense vector, the memory requirements can increase up to several gigabytes. During training, since these large datasets are frequently accessed, the locality of the training dataset is important. These data volumes are too large to fit into the internal memory of a GPU, so multiple external dynamic random access memories (DRAMs) need to be used. Each DRAM chip communicates with the processor through an electronic communication carrier. For example, in a computing system that utilizes a silicon interposer, the processor and memory chips are attached on the same interposer, and the communication between the memory and the processor is supported by conductive traces formed on the interposer. In recent years, the use of silicon interposers has become widespread because the density of conductive traces that the interposer can provide is much greater than that in a printed circuit board (PCB). However, interposers cannot scale infinitely. Interposers manufactured using microfabrication technology have a limited area, and thus the number of memory chips that can be accommodated by the interposer is limited.

[0042] Moreover, the presence of parasitic impedance further limits the bandwidth scalability. It would be ideal if the impedance of the conductive traces of the interposer could be ignored. In reality, unfortunately, the impedance of the traces is quite large. Parasitic impedance limits the bandwidth scalability in two ways. First, it limits the bandwidth that the traces can support. Second, it increases the power consumption. Even worse, the parasitic impedance increases with the length of the traces. That is, the greater the separation distance between the memory chip and the processor, the narrower the bandwidth. For this reason, conventional computing systems are usually designed such that the memory chip is positioned within a few centimeters of the processor. However, only a limited number of chips can be accommodated within this range. As a result, conventional computing systems are limited in both memory bandwidth and memory capacity.

[0043] II. Overview of the Photonic Communication Platform The communication platform developed by the inventors overcomes these limitations by using photonics. Due to the physical properties of light propagating inside the waveguide, optical communication is inherently immune to the influence of parasitic impedance. Being immune to the influence of parasitic impedance brings a great benefit that there is no need to position the memory chip within a specific range of the processor.

[0044] Another advantage of the photonic communication platform developed by the inventors is that it can be easily adapted to various computer architectures. The architecture of a single-node computer includes one processor chip (which may have two or more processor cores per chip) and multiple memory chips. The architecture of a multi-node computer includes multiple processor chips and multiple memory chips. Some multi-node architectures use a ring topology, where each processor communicates directly with two neighboring processors, and communication with other processors is via the neighboring processors. Other multi-node architectures use a star topology, with a central hub responsible for routing communication between cores. Yet another multi-node architecture uses a multicast topology, where each processor communicates directly with multiple other processors.

[0045] Some aspects of the photonic communication platforms described herein enable easy adaptation to any of these architectures (and other architectures). Some embodiments provide a photonic communication platform that uses "photonic modules". Each photonic module includes a programmable photonic circuit that can be configured based on the needs of a particular computer architecture. Some platforms are arranged according to a one-dimensional scheme, such as blocks of 3x1 modules, blocks of 5x1 modules, 10x1 modules, blocks of 20x1 modules, and so on. Some platforms are arranged according to a two-dimensional scheme, such as blocks of 3x3 modules, blocks of 5x3 modules, blocks of 5x5 modules, blocks of 10x10 modules, and so on. More generally, the platform can be any NxM module block and any topology (such as T-topology, L-topology, X-topology, etc.) as long as N≥1 and M≥1. Each photonic module can function as a node of a computing system. Each node may have one or more digital processor chips, one or more analog accelerators, one or more photonic accelerators, one or more memory chips, one or more networking chips, or other devices.

[0046] FIG. 1 illustrates an example of a computing system based on a photonic communication platform in which nine photonic modules are arranged in a 3x3 topology. Computing system 10 includes a photonic substrate 20 patterned with nine photonic modules 22 (also referred to herein as “photonic sites” or simply “sites”). This photonic communication platform supports one processor die (30) positioned at the center of the photonic substrate 20 and eight memory nodes surrounding the processor die. Some of the memory nodes include a single memory chip (see, for example, memory die 32). Other memory nodes include stacked memories that include multiple vertically stacked memory dies (see, for example, stacked memory 34). The dies can communicate electronically (e.g., using through-silicon vias, copper pillars, microbumps, BALL GRID ARRAY (registered trademark: registration number 3370248) or other electrical interconnections) and / or optically (e.g., using grating couplers, prisms, lenses or other optical couplers) with the photonic modules.

[0047] As will be described in detail later, the photonic module is patterned with optical waveguides and an optical distribution network. The optical distribution network of the photonic module can optically communicate the die of that particular photonic module selectively with any other die of the computing system. For example, the optical distribution network of the photonic module positioned under the processor die 30 may be reconfigured according to the needs of the processor. At the start of a routine, the processor may need to access data stored in the first memory node. This read operation involves configuring the respective optical distribution network to optically communicate the processor with the first memory node. Thereafter, during the routine, the processor may need to write data to the second memory node. This write operation involves reconfiguring the optical distribution network to optically communicate the processor with the second memory node.

[0048] The inventors further understand that manufacturing photonic modules on a large scale can be costly. The photonic communication platforms described herein are engineered to limit manufacturing costs. These platforms rely on the use of a common photomask set (or at least one common photomask) to manufacture multiple photonic modules. This approach reduces costs in two ways. First, it reduces the additional costs incurred when procuring multiple different photomask sets. Second, while some standard semiconductor foundries require the use of the same photomask set (or at least one photomask) across the entire wafer, this approach enables the use of those foundries to manufacture photonic modules. Designing photonic modules that share at least one photomask enables the manufacture of multiple photonic modules on the same semiconductor wafer while leveraging standard low-cost step-and-repeat manufacturing processes.

[0049] III. Photonic Module The photonic module described in this specification may be manufactured using a microfabrication technology including, for example, complementary metal oxide semiconductor (CMOS) microfabrication technology. Accordingly, some embodiments relate to a silicon photonic-based optical communication platform. Some specific microfabrication technologies include a step-and-repeat approach, by which a semiconductor wafer is patterned with multiple copies of a template layout using a stepper machine. FIGS. 2A through 2E illustrate a microfabrication technology for manufacturing a photonic module. FIGS. 3A through 3F illustrate examples of photonic modules patterned using these microfabrication technologies.

[0050] Referring first to FIG. 2A, this figure illustrates a semiconductor wafer 100. Wafer 100 may be made of any material. For example, wafer 100 may be made of silicon (or otherwise include silicon). In one example, wafer 100 is a silicon-on-insulator (SOI) wafer. In another example, wafer 100 is a bulk silicon wafer. Wafer 100 may have any size. For example, the diameter of wafer 100 may be, among other values, 150 mm, 300 mm, or 450 mm. However, not all wafers need to have a circular shape.

[0051] FIG. 2B illustrates a set of photomasks that may be used to pattern wafer 100 using photolithography technology. Photomask set 200 includes three photomasks (201, 202, and 203), although other sets may include a greater or lesser number of photomasks. Each photomask has a specific pattern consisting of opaque regions and transparent regions. When a photomask is exposed, the opaque regions block light, thereby preventing the wafer from being illuminated, and allowing light to pass through the transparent regions. As a result, the pattern of the photomask is transferred to the wafer.

[0052] Each photomask may define a particular layer of the photonic module. One photomask may be used to define an optical waveguide. When the wafer undergoes an etching process, only the exposed areas (or only the unexposed areas) are etched away, and the other areas remain unetched. This photomask may be patterned to form a network of optical waveguides when the wafer is exposed through this photomask. FIG. 2C illustrates a portion of a photomask that may be used to form waveguides on wafer 100. The lines of photomask 201 represent opaque regions. The background of photomask 201 is transparent. When photomask 201 is exposed so that an image of the photomask is projected onto wafer 100, it becomes possible to pattern the optical waveguide in the shape of the opaque regions. In this particular example, the pattern of the lines of the photomask becomes a grid of optical waveguides.

[0053] Some photonic modules involve the use of different levels of optical waveguides. In some such embodiments, the photomask set 200 may include a dedicated photomask for each waveguide level. Another photomask may be used to define the n-doped regions. When the wafer undergoes an ion implantation or dopant diffusion process, only the exposed regions (or only the unexposed regions) are doped, and the other regions remain undoped. Another photomask may be used to define the p-doped regions using a similar process. Some photonic modules also involve the use of different doping concentrations. In some such embodiments, the photomask set 200 may include a dedicated photomask for each doping concentration. In other embodiments, the photomask set 200 may include a photomask used to define the deposition of semiconductor materials other than silicon, such as germanium, and / or other materials from the periodic table, such as group III or group V materials. Another photomask may be used to define the metal contacts. Another photomask may be used to define the metal traces. Some photonic modules involve the use of different levels of metal traces. In some such embodiments, the photomask set 200 may include a dedicated photomask for each metal trace level.

[0054] In some embodiments, the wafer 100 is patterned in a step-and-repeat manner. When the wafer 100 is processed by a stepper machine, the pattern of the photomask is repeatedly exposed in a grid pattern across the entire surface of the wafer. This process involves stepping the wafer back and forth and side to side under the lens of the stepper and exposing the photomask at each step. As a result, the wafer 100 will be patterned with multiple copies of the pattern defined by the photomask. This operation may be repeated for each photomask in the set (or at least for some of the photomasks).

[0055] In some embodiments, this process may be used to pattern the wafer 100 with multiple copies of the template photonic module. In the example of FIG. 2D, the wafer 100 is patterned with a lattice of photonic modules 22. The photonic modules may share the pattern of one or more of the photomasks of set 200. For example, the photonic modules may share the pattern of the same waveguide photomask and / or the same metal trace photomask. In other embodiments, the photonic modules share the pattern of all of the photomasks of set 200. For example, the photonic modules may share the same optical waveguide pattern, the same n-doped pattern, the same p-doped pattern, the same contact pattern, the same metal trace pattern, and so on.

[0056] In some embodiments, the entire surface of the wafer 100 is patterned using the photomask set 200. However, not all embodiments are limited in this regard. This is because a portion of the wafer 100 may be patterned using a first photomask set and another portion of the wafer 100 may be patterned using a second photomask set. In some embodiments, the first photomask set and the second photomask set may share one or more common photomasks, such as a waveguide photomask.

[0057] Once patterned, the wafer 100 may include a plurality of photonic substrates. The photonic modules 22 may be separated together from the wafer to form photonic substrates of any desired shape and size. For example, the wafer of FIG. 2E is marked to obtain six photonic substrates from the wafer 100. This figure identifies a 1x1 photonic substrate having only one photonic module 22, a 2x2 photonic substrate having four photonic modules 22, a 2x3 photonic substrate having six photonic modules 22, and three 3x3 photonic substrates each having nine photonic modules 22. Separating the photonic substrates from the wafer involves dicing the wafer along the outer perimeter of the desired photonic substrate. One of the 3x3 photonic substrates of the wafer 100 may be used as the photonic substrate (see photonic substrate 20) in the example of the computing system of FIG. 1.

[0058] With the techniques described with respect to FIGS. 2A - 2D, the manufacture of photonic modules can be made relatively low - cost. Some semiconductor foundries require the use of the same photomask set (or at least one photomask) to pattern the entire wafer (or at least a portion of the wafer). Otherwise, using different photomasks to pattern different portions of the wafer would require replacing the photomask with another mask between photolithography exposures, making the step - and - repeat process inefficient and expensive. Designing photonic modules that share at least one photomask enables the manufacture of multiple photonic modules on the same semiconductor wafer while leveraging the standard low - cost step - and - repeat process.

[0059] FIG. 3A illustrates an example of the photonic module 22. In this example, the photonic module 22 is shaped as a rectangle (although other shapes are also possible, such as a square or other polygon). Therefore, the photonic module 22 is bordered by four boundary lines (boundary lines 1, 2, 3, and 4). Boundary line 1 is opposite to boundary line 2, and boundary line 3 is opposite to boundary line 4. Boundary line 1 is adjacent to boundary lines 3 and 4, and boundary line 2 is also adjacent to boundary lines 3 and 4. The photonic module 22 includes an optical distribution network 104 coupled to waveguides 111, 112, 113, and 114. The waveguide 111 optically couples the optical distribution network 104 to the boundary line 1. Therefore, an optical signal coupled from the optical distribution network 104 to the waveguide 111 can be transferred outside the photonic module by crossing the boundary line 111. Similarly, the waveguide 112 optically couples the optical distribution network 104 to the boundary line 2, the waveguide 113 optically couples the optical distribution network 104 to the boundary line 3, and the waveguide 114 optically couples the optical distribution network 104 to the boundary line 4. In some embodiments, the boundary lines of the photonic module are defined based on photolithography shots (e.g., the boundary lines are defined by the boundary lines of the photomask used to manufacture the photonic module). However, in other embodiments, one photolithography shot may define two or more photonic modules. For example, the photomask may be patterned with a plurality of adjacent instances of a template photonic module. In some such embodiments, the boundary lines of the photonic module are defined where adjacent instances of the template photonic module meet.

[0060] The example of FIG. 3A illustrates waveguides coupling an optical distribution network to each of the boundaries, but not all embodiments are arranged in this way. In other embodiments, the photonic module 22 may include two of these four waveguides, for example, waveguides 111 and 112, or waveguides 111 and 113. In still other embodiments, the photonic module 22 may include three of these four waveguides, for example, waveguides 111, 112, and 113. The optical distribution network 104 includes photonic components (e.g., photonic switches) for routing optical signals inside and outside the photonic module 22.

[0061] In some embodiments, the photonic module may include a photonic waveguide composed of multiple layers. Similar to how a conductive trace composed of multiple layers enhances the electrical signal routing ability of an electronic circuit, a waveguide composed of multiple layers enhances the optical signal routing ability of the photonic module. In one example, one layer includes a silicon waveguide and one layer includes a silicon nitride waveguide. In another example, multiple layers include silicon waveguides. In addition to, or instead of, this, multiple layers may include silicon nitride waveguides. The selection of the material for each waveguide layer may be determined by the wavelength of the light that will be routed by the waveguide. For example, the silicon layer and the silicon nitride layer may be used to route infrared light in the telecommunications band at a wavelength of 1.3 μm or 1.5 μm. In some examples, the waveguide composed of multiple layers may also include an aluminum nitride waveguide that can be used to route visible light up to the UV wavelength, or an aluminum oxide waveguide that can be used to route UV light. Each layer may be arranged in a configuration having an optical distribution network that routes signals between the waveguides of the layer, similar to the configuration illustrated in FIG. 3A.

[0062] The photonic module 22 further includes one or more out-of-plane couplers 105. The waveguide 117 optically couples the out-of-plane coupler 105 to the optical distribution network 104. The out-of-plane coupler 105 is configured to emit the light received from the waveguide 117 outside the xy plane, for example, in a direction parallel to the Z axis or at an angle with respect to the Z axis. The out-of-plane coupler 105 may be further configured to capture the light illuminating from outside the xy plane and transmit the captured light to the waveguide 117. The out-of-plane coupler 105 enables optical communication between the photonic module 22 and a die provided above and / or below the photonic module. The out-of-plane coupler 105 may be implemented using any suitable optical components including, for example, an optical grating, a lens, and a prism. In some embodiments, the optical distribution network may be configured such that the same out-of-plane coupler enables optical communication in both the direction from the optical distribution network 104 to the die and the direction from the die to the optical distribution network 104. In other embodiments, one out-of-plane coupler 105 may enable optical communication in one direction, and another out-of-plane coupler 105 (not shown in FIG. 3A) may enable optical communication in the reverse direction. In one embodiment, the out-of-plane coupler 105 may be used to couple a light source to the optical distribution network 104. The light source can be one of a laser (continuous wave or pulsed), an LED, or a superluminescent diode.

[0063] Figure 3B illustrates how the out-of-plane coupler 105 can be used to enable out-of-plane optical communication. For ease of understanding, only the out-of-plane coupler 105, waveguide 117, and optical distribution network (ODN) 104 are shown inside the optical module 22. In this example, the out-of-plane coupler 105 is implemented with an optical grating. Die 320 is attached to the photonic module 22. Die 320 may include a processor, memory, and / or other electronic components (not shown in Figure 3B). Further, die 320 includes an out-of-plane coupler 351, waveguide 317, and controller 322. Controller 322 is electrically coupled to the optical distribution network 104 via electrical connection 324, which may include, for example, BALL GRID ARRAY (registered trademark: registration number 3370248), copper pillars, through-silicon vias, microbumps, metal pads, etc. In this example, the out-of-plane coupler 105 emits light in a direction parallel to the Z-axis towards the out-of-plane coupler 351. The out-of-plane coupler captures the light and transmits the captured light to the controller 322 via waveguide 317.

[0064] Controller 322 controls the operation of the optical distribution network 104. For example, controller 322 controls the state of the switches in the optical distribution network 104. Control signals are supplied to the optical distribution network 104 via electrical connection 324. Alternatively, or in addition to this, the controller can be formed directly on the photonic module 22, and this controller may control the operation of the optical distribution network 104. This controller may supply control signals to the optical distribution network 22 via conductive traces formed on the photonic module 22.

[0065] Referring back to FIG. 3A, the optical distribution network 104 may selectively couple any component of the photonic module 22 to any other component of the photonic module 22. For example, the optical distribution network 104 may allow light to pass between waveguide 111 and waveguide 112, and / or between waveguide 111 and waveguide 113, and / or between waveguide 113 and waveguide 114, and / or between the out-of-plane coupler 105 and waveguide 111, and / or between the out-of-plane coupler 105 and waveguide 113, and so on.

[0066] FIG. 3C illustrates how the out-of-plane coupler 105 can be used to communicate between two photonic communication fabrics. For ease of understanding, the figure shows only two photonic modules 22, with one from each photonic communication fabric optically coupled to each other using the out-of-plane coupler 105. The controller 322 is electrically coupled to both optical distribution networks 104 using electrical connections 324 and silicon through vias 125. Stacking multiple photonic communication fabrics on top of each other increases the number of optical and electronic communication channels between each site. Further, having multiple communication fabrics can reduce the number of waveguide crossings required to route optical signals across the photonic module, thereby reducing optical losses and improving the overall power budget.

[0067] The photonic substrate may include a plurality of photonic modules connected to each other and collectively form an optical network. FIG. 3D illustrates an example of a 2x3 photonic substrate including six photonic modules 22. This photonic substrate is obtained by dicing and separating a group of 2x3 photonic modules from a wafer 100 (see FIG. 2E). The photonic modules 22 are arranged such that the waveguide 111 of an optical module is aligned with the waveguide 112 of the optical module on the left side of that optical module, the waveguide 112 of an optical module is aligned with the waveguide 111 of the optical module on the right side of that optical module, the waveguide 113 of an optical module is aligned with the waveguide 114 of the optical module above that optical module, and the waveguide 114 of an optical module is aligned with the waveguide 113 of the optical module below that optical module. As a result, the optical modules form an optical network. The optical distribution network 104 may route optical signals anywhere inside and outside the network. For example, assume that a processor is attached to a photonic module positioned at the northwest corner of the photonic substrate and a memory is attached to a photonic module positioned at the southeast corner of the photonic substrate. The read operation may involve reconfiguring each optical distribution network to optically communicate the processor with the memory.For example, an optical communication path is formed such that 1) a processor is coupled to an out-of-plane coupler of a photonic module to which the processor is attached, 2) the out-of-plane coupler of the photonic module is coupled to a waveguide 112 of the same photonic module, 3) the waveguide 112 of the photonic module is coupled to a waveguide 111 of an adjacent photonic module (the uppermost central photonic module), 4) the waveguide 112 of the uppermost central photonic module is coupled to a waveguide 111 of the next adjacent photonic module (the northeast corner of the photonic substrate), 5) the waveguide 114 of the photonic module positioned at the northeast corner is coupled to a waveguide 113 of a photonic module to which a memory is attached, and 6) the waveguide 113 of the photonic module to which the memory is attached is coupled to an out-of-plane coupler of the same photonic module.

[0068] As described above, the waveguides of adjacent photonic modules are optically coupled to each other, thereby enabling light to pass from one photonic module to the next. In some embodiments, the waveguides may be physically connected. This arrangement is illustrated in FIG. 3E, which depicts the region at the boundary between two adjacent photonic modules. As shown, the waveguide 112 of the photonic module positioned on the left is physically connected to the waveguide 111 of the photonic module positioned on the right. In some embodiments, a continuous waveguide extends across the boundary line between the respective optical distribution networks of the photonic modules.

[0069] In other embodiments, there may be a gap between the waveguides. This arrangement is shown in FIG. 3F. In this example, each waveguide has an end portion positioned at a distance from the boundary line. Thus, a gap is formed in the boundary region. Despite the gap, waveguides 111 and 112 are still optically coupled to each other. In this case, in practice, the light emitted at the end of one waveguide reaches the end of the other waveguide by free-space propagation. If the size of the gap is small enough (e.g., less than 500 μm), most of the optical power radiated by one waveguide couples to the other waveguide.

[0070] In yet other embodiments, as shown in FIG. 3G, a photonic bridge may be used to optically couple the waveguides to each other. In this example, the ends of the waveguides are coupled to respective out-of-plane couplers 152. The photonic bridge die 300 is attached to the boundary region. The photonic bridge die 300 includes a pair of out-of-plane couplers 352 and an optical waveguide 354 that couples the out-of-plane couplers to each other. Assume that the processor die 302 needs to send a read message to the memory die 304. This can be achieved by: 1) transmitting light out-of-plane from the processor die 302 to the respective photonic modules (e.g., in the manner shown in FIG. 3B), 2) transmitting the light to waveguide 112, and as a result, transmitting the light to out-of-plane coupler 152, 3) transmitting the light to out-of-plane coupler 352, 4) transmitting the light to optical waveguide 354, and as a result, transmitting the light to the other out-of-plane coupler 352, 5) transmitting the light to the other out-of-plane coupler 152, and 6) transmitting light out-of-plane from that photonic module to the memory die 304 (e.g., in the manner shown in FIG. 3B).

[0071] In some embodiments, the photonic module 22 may be patterned according to a common metal trace photomask. As a result, the photonic modules share the same pattern of metal traces. In some embodiments, the photonic module 22 is patterned according to a plurality of common photomasks. As a result, multiple levels of metal traces share the same pattern across different photonic modules. Some of the metal traces may be used to deliver power across the photonic substrate. Some of the metal traces may be used to deliver electronic signals across the photonic substrate.

[0072] FIG. 3H illustrates a 2x3 photonic substrate where each photonic module 22 shares the same pattern of metal traces. For illustrative purposes, only the metal traces are shown in this figure, but each photonic module further includes, for example, in the arrangement shown in FIG. 3D, waveguides, one or more out-of-plane couplers, and an optical distribution network. In this example, there are two levels of metal traces. Each level of metal traces is fabricated using the same photomask across different photonic modules. The metal traces of metal trace level 1 extend horizontally, thereby electrically coupling photonic modules adjacent to each other in the horizontal direction. The metal traces of metal trace level 2 extend vertically, thereby electrically coupling photonic modules adjacent to each other in the vertical direction. Of course, other arrangements are also possible. For example, in other embodiments, the metal traces of the same level may electrically couple one photonic module to all photonic modules adjacent to that photonic module.

[0073] The metal traces are arranged to convey electricity (e.g., signals and / or power) across the boundary of the photonic module. This may be achieved by patterning the metal traces to be continuous across the boundary of the photonic module. In this example, the level 1 metal traces are continuous across the vertical boundary, and the level 2 metal traces are continuous across the horizontal boundary. Metal traces of different levels may be connected to each other using vias (not shown in FIG. 3H). In some embodiments, the photonic modules may share the same pattern of vias. In other words, the same via photomask may be used for each photonic module. In some embodiments, the photonic module may have a larger number (tens to hundreds) of metal traces. Some of these metal traces may be arranged to be continuous across the photonic module, but in some embodiments, most of the metal traces need not be patterned to be continuous across the module.

[0074] The metal traces may be used to deliver power and / or electrical signals across the photonic substrate. In one example, a power source is connected to a particular photonic module. The power generated by the power source may be delivered from that particular photonic module to other photonic modules using the metal traces. In another example, a controller chip may be bonded (e.g., 3D bonded) to a particular photonic module. The control signal generated by the controller may be delivered from that particular photonic module to other photonic modules using the metal traces. The control signal may control the state of the optical distribution network of the photonic module.

[0075] As described above, an electronic control circuit may be used to control the operation of the photonic module. These electronic control circuits may, for example, control how the optical distribution network 104 routes optical signals. The electronic control circuit may be integrated with the photonic module in various ways. In some embodiments, the photonic module may be formed on a first substrate and the electronic control circuit may be formed on a second substrate. The two substrates may be joined to electrically communicate the electronic control circuit with the optical distribution network. However, in other embodiments, the electronic control circuit may be manufactured directly on the same substrate as the photonic module. Manufacturing the photonic module and the electronic control circuit on the same substrate may reduce costs because it eliminates the need to rely on two separate manufacturing processes and one bonding process, and instead only one manufacturing process may be required.

[0076] Figure 3I is a cross-sectional view of a photonic substrate integrating a photonic module with transistors. The transistors may be connected to each other to define an electronic control circuit. In this example, the photonic substrate is formed on a SOI substrate, although other types of substrates including bulk silicon substrates are also possible. An insulating layer (e.g., a silicon dioxide layer) is formed on the silicon substrate. A silicon layer is formed on the insulating layer. The silicon layer is patterned to form other optical components such as waveguides and the components described with respect to FIG. 3A. This cross-sectional view illustrates a portion of a Mach-Zehnder interferometer in which an arm is defined by waveguides 370 and 371. This Mach-Zehnder interferometer defines one of the switches of the optical distribution network 104. Transistor block 380 is formed in the same silicon layer as waveguides 370 and 371. Transistor block 380 includes a plurality (e.g., tens of thousands, hundreds of thousands, millions, or more) of transistors that are all connected to form an electronic control circuit. The photonic substrate further includes multiple levels of metal traces (only two levels of metal traces are shown in this example). Vias connect the metal traces to the waveguides and transistors. The metal traces enable the electronic control circuit to control the operation of the Mach-Zehnder interferometer.

[0077] FIG. 4 is a cross-sectional view of an example of a computing system 400 based on a photonic substrate 20 (e.g., a 3x3 photonic substrate). A die stack including dies 420, 421, and 422 is attached to a photonic module positioned on the left side of the photonic substrate 20. These dies may form, for example, a stacked memory unit. A laser die 430 is attached to one side of the photonic module in the center of the substrate, and a die 431 is attached to the opposite side of the same photonic module. To support the opposite die, the photonic module may include at least one out-of-plane coupler that emits light in an upward direction and at least one out-of-plane coupler that emits light in a downward direction. Dies 440 and 441 are attached next to each other in the same photonic module. Dies 440 and 441 may include, for example, a processor or memory. As described with respect to FIG. 3D, the photonic module provides a platform for distributing optical signals from one die to another.

[0078] The laser die 430 includes one or more lasers. The light generated by the lasers can be distributed throughout the computing system and can serve as a reference light modulated with data. The laser die 430 can include a III-V laser, such as an InP-based laser. The laser die 430 can be bonded to a photonic substrate, for example, using surface mount technology. The lasers of the laser die 430 can be coupled to the semiconductor substrate using an out-of-plane coupler. In some embodiments, the laser light emitted in a direction parallel to the chip surface can be directed toward the out-of-plane coupler using a ball lens.

[0079] Recent progress in die - wafer bonding of III - V lasers onto silicon - photonic - based wafers indicates that the yield of this process can be significantly below 100%. To avoid this problem, two or more lasers can be directed towards the same input of the photonic substrate. One laser at a time can be used, but if one laser fails, another laser can be turned on and fed into the photonic substrate. Having multiple lasers per photonic substrate increases the reliability of the platform in case one or more lasers fail.

[0080] The options for external light sources in optical communication platforms are not limited to chip - based III - V lasers. Other lasers (e.g., semiconductor - based ones as used in optoelectronic communications) can be coupled to the platform using optical fibers or free - space optics such as lenses. In some embodiments, a 1 - to - N type splitter can be used to reduce the number of laser dies as a single laser can supply light to multiple photonic modules.

[0081] Depending on the application, multiple lasers may be required, and it may be further necessary for the optical signals emitted by the lasers to be coherent with each other (e.g., temporally coherent). In some such embodiments, an optical communication platform of the type described herein can be used to lock the phase of one or more lasers to a single master laser. One or more optical distribution networks 105 may be configured such that the master laser is mixed with one of the slave lasers and the beat interference fringes are measured. The beat interference fringes are used as an error signal for locking the phase so that the entire system is coherent.

[0082] Lasers emitting at different wavelengths may, in some embodiments, be used to support a wavelength multiplexing scheme. For example, a wavelength division multiplexing (WDM) scheme can be used to increase the utilization of the bandwidth per waveguide. Other schemes include multimode waveguides, time division multiplexing, and / or polarization diversity. These techniques support multiple independent communication channels using the same optical path.

[0083] In some embodiments, instead of the aforementioned lasers, a suitable light source other than a laser, such as an LED or a superluminescent diode, may be used. The choice of the light source is also motivated by the choice of the wavelength of the photonic communication fabric. If the fabric is intended to communicate using visible light, the light source must be selected to output light at an appropriate wavelength.

[0084] IV. Optical Distribution Network The optical distribution network 104 may be implemented using optical switches. Examples of optical switches include Mach-Zehnder interferometers, optical resonators, multimode interference (MMI) waveguides, arrayed waveguide gratings (AWGs), thermo-optic switches, acousto-optic switches, magneto-optic switches, MEMS optical switches, non-linear optical switches, liquid crystal switches, piezoelectric beam steering switches, grating switches, dispersion switches, and so on.

[0085] The optical distribution network 104 may be static or dynamic (e.g., reconfigurable based on an electrical control signal or an optical control signal). A static network may, for example, receive multiple wavelengths from the same input waveguide and route each wavelength to a different output waveguide. Another static network may receive two orthogonal polarizations from the same input waveguide and route each polarization to a different output waveguide. Another static network may receive multiple modes from the same input multimode waveguide and route each mode to a different output waveguide.

[0086] A dynamic optical distribution network may be reconfigured according to the needs of a computing system. FIG. 5A illustrates an example of a dynamic optical distribution network. In this example, the optical distribution network 104 includes a 3x1 switch 602, two 1x2 switches 606, and an optical coupling element 107. This photonic module further includes waveguides 111-114 and an out-of-plane coupler 105, which may enable optical communication with a laser die or other die. The switch 602 selects one of the waveguides 111, waveguide 113, and the out-of-plane coupler 105 as an input. The first switch 604 routes the input received from the switch 602 to either the optical coupling element 107 or the switch 604. The optical coupling element 107 directs the optical power to the photonic transmitter illustrated in FIG. 6. In some embodiments, the optical coupling element 107 includes an out-of-plane coupler for coupling light to the die. The second switch 604 routes the input received from the first switch 604 to either waveguide 112 or waveguide 114. In some embodiments, the switch 604 is implemented using a Mach-Zehnder interferometer.

[0087] FIG. 5B illustrates a 3x3 photonic substrate including a photonic module of the type illustrated in FIG. 5A. In this example, the photonic chip positioned at the northwest corner of the photonic substrate is coupled to a laser 600. In some embodiments, the laser 600 is formed on a laser die and optically coupled to an out-of-plane coupler 105. In other embodiments, the laser 600 is integrated as part of its photonic module (e.g., disposed in a trench formed through the top surface of the photonic module). The laser 600 may emit a single wavelength or multiple wavelengths. In some embodiments, the laser 600 supplies light to the entire photonic substrate, although in other embodiments, other photonic modules may also have lasers.

[0088] FIGS. 5C and 5D illustrate other possible implementations of the optical distribution network 104. The example of FIG. 5C is a fully connected implementation where all the boundary lines of the photonic modules are coupled to each other. The light entering from the boundary lines passes through a plurality of 1x2 switches 604, which determine whether the light travels straight, left, or right. In some embodiments, the routing may be performed for each optical channel (e.g., each waveguide mode, polarization, or wavelength).

[0089] However, a fully connected routing topology may not be necessary or may not be feasible in some embodiments. To reduce the complexity of the optical distribution network, the routing options may be restricted to fewer, as shown in the example of FIG. 5D (including two 1x2 switches). Reducing the routing options makes it possible to reduce the number of switches per optical module, thereby reducing power consumption and channel crosstalk and improving the signal-to-noise ratio (SNR). However, these gains are achieved at the expense of data bandwidth.

[0090] V. Optical Interconnection Fabric FIG. 6A illustrates an example of a reconfigurable photonic communication fabric. This communication fabric includes a plurality of switches. Note that "2:2" indicates a 2x2 switch and "3:3" indicates a 3x3 switch. The switches may be configured according to the needs of the computing system. Transceiver 700 includes an opto - electrical converter and an electro - optical converter. In some embodiments, transceiver 700 embodies the optical coupling element 107 of FIG. 5A. FIG. 6B illustrates an example of transceiver 700. For ease of understanding, only one transmitter / receiver pair is shown, but there may be a transmitter / receiver pair for each waveguide coupled to transceiver 700. Switch 702 arbitrates between transmitter (TX) 704 and receiver (RX) 706. TX704 includes an electro - optical converter such as an optical modulator. RX706 includes an opto - electrical converter such as an optical receiver. FIG. 6C illustrates a 3x3 photonic substrate, and each node includes the photonic module of FIG. 6A.

[0091] VI. Electronic Switching Network The inventors recognize that interfacing a die (e.g., a memory, a processor, etc.) with the photonic substrates described herein can present compatibility challenges. Ideally, the die is pre-specified by a control circuit for controlling the operation of the optical distribution network and pins that are perfectly aligned with the pins of the photonic substrate. In this way, once the die and the photonic substrate are joined, they tend to be pre-provisioned to communicate easily with each other. However, since the die and the photonic substrate are often manufactured by different entities, this approach can be impractical. For example, a U.S. entity may manufacture the photonic substrate and a separate Japanese entity may manufacture the die. This approach can significantly increase the cost for the die manufacturer as it places the burden of including control circuits and pins that are compatible with the photonic substrate on the entity manufacturing the die. Recognizing this problem, the inventors have developed an electronic switching network that functions as an interface between the photonic substrate and the die. These electronic switching networks are arranged so that there is no need to re-design the die to be compatible with the photonic substrate, thus saving costs for the die manufacturer. In short, the electronic switching network defines the switching protocol and the control protocol of the photonic communication platform.

[0092] Some electronic switching networks are manufactured with photonic modules. In some embodiments, the electronic switching network is formed on the same substrate on which the photonic module is formed. Referring back to FIG. 3I, for example, the electronic switching network may be defined by transistors 380. In other embodiments, the photonic module is formed on a first substrate, the electronic switching network is formed on a second substrate, and the substrates are bonded (e.g., 3D bonded). Regardless of how it is formed, the electronic switching network may be used to program the optical distribution network 104. By using the electronic switching network, it is possible to design a die with minimal interface circuitry, reducing the costs that the die manufacturer must bear to make the die compatible with the photonic substrate.

[0093] FIG. 7A illustrates a photonic substrate integrated with (e.g., directly bonded or formed on the same substrate as) an electronic switching network. This photonic substrate includes six photonic modules arranged in a 2x3 configuration. As described above, each photonic module includes an optical distribution network 104. The electronic switching network includes a plurality of controllers 740, a digital-to-analog converter (DAC) 750, a memory 742, and a debugging unit 744. Each controller 740 controls the optical distribution network through the DAC 750. The controller 740 may control the state of the switches of the optical distribution network so as to dynamically reconfigure the optical links as needed. Referring back to the example of FIG. 6C, the controller 740 may be used to control the state of the 2x2 switch and the state of the 3x3 switch.

[0094] The electronic switching network may program the optical distribution network based on the data stored in the memory 742. For example, the memory may store instructions that, when executed, cause the optical distribution network to execute a pre-defined switching sequence. Alternatively, the memory may store instructions that, when executed, cause the optical distribution network to dynamically optimize the optical links based on the needs of a particular application. In some embodiments, the electronic switching network may monitor the use of the optical links and determine which optical links are in use, which are available, or which can provide a bandwidth exceeding a threshold bandwidth. The electronic switching network may use this information to determine how to allocate the optical links to specific data streams. This may be particularly useful, for example, in a photonic communication platform that connects hundreds of memory dies and processors. Such a photonic communication platform can rely on the electronic switching network to determine the optimal optical path that allows a particular processor die to access information from a particular memory die. In some embodiments, a machine learning algorithm may be used to identify the optimal optical links.

[0095] In some embodiments, the electronic switching network is configured to perform the following steps. At a first time, the electronic switching network programs the optical switches of the photonic substrate to form a first optical communication path that couples a first subset of the plurality of dies to each other (e.g., couples a first die to a second die). Programming the optical switches at the first time may involve changing the state of some of the switches (e.g., a first subset of the switches) without changing the state of other switches, or may involve changing the state of all the switches. At a second time following the first time, the electronic switching network programs the optical switches to form a second optical communication path that couples a second subset of the plurality of dies to each other (e.g., couples a third die to the first die, or couples a third die to a fourth die). The electronic switching network may continue to reprogram the optical switches as needed. Programming the optical switches at the second time may involve changing the state of some of the switches (e.g., a first subset of the switches or a second subset of the switches) without changing the state of other switches, or may involve changing the state of all the switches. Changing the state of a switch may involve any of the following operations: changing the output of the switch from an off state to an on state; changing the output of the switch from an off state to a partially on state (e.g., a state where the switch delivers a portion of the input power to the output and a portion of the input power to one or more other outputs); changing the output of the switch from an on state to a partially on state; changing the output of the switch from an on state to an off state; selecting one or more inputs of a switch that were not previously selected; and / or deselecting one or more inputs of a switch that were previously selected.

[0096] In the example of FIG. 7B, the electronic switching network programs the photonic substrate to form an optical path from the first die to the second die. Then, as shown in FIG. 7C, the electronic switching network reprograms the photonic substrate to form an optical path from the third die to the first die. In some embodiments, the electronic switching network can utilize wavelength division multiplexing and / or time division multiplexing to program the optical distribution network.

[0097] A typical connection between nodes may start from a request message on the electronic switching network. The request may perform arbitration through a mesh of electronic switches to establish the connection. At each network hop, when an electrical request wins the arbitration, an optical link is created. When the connection is fully established between the source and the destination, an acknowledgment is sent back to the requester on the electronic network, and data transmission proceeds over the optical link.

[0098] The controller 740 may use the debugging unit 744 to perform tests, diagnostics, and isolation of defects in metal traces, contacts, pins, pads, etc. In some embodiments, the debugging unit is implemented using the industry standard of the Joint Test Action Group (JTAG). In some embodiments, due to manufacturing defects, the communication channel may malfunction. By being able to map redundant connections (optical and / or electrical) as an alternative, it may not be necessary to discard platforms with few defects.

[0099] By being able to dynamically reconfigure the photonic communication platform in a dynamic manner, it becomes possible to operate the electronic switching network with a relatively low bandwidth, and as a result, the power consumption becomes relatively low. The low power consumption is particularly beneficial when the amount of data transmitted on the photonic module during a switching event is relatively large. In some embodiments, the low power consumption opens up the opportunity for "always-on" operation that can monitor the training or reconfiguration sequence.

[0100] In some embodiments, the power and fidelity of an optical signal propagating across multiple photonic modules may depend on the number of waveguide crossings where the signals cross. Thus, the farther the optical signal needs to travel, the greater the power consumption and the lower the fidelity. Recognizing this problem, the inventors have developed a configuration that can adaptively select an encoding scheme that uses uppercase and lowercase alphabets (whether the number of bits per symbol is large or small) according to the number of boundaries of the photonic modules where the signal is expected to cross, or the expected optical loss, or any other characteristic of the optical path. For example, when few crossings or low loss are expected, an encoding scheme using uppercase alphabets can be used. Examples of such encoding schemes include quadrature amplitude modulation (QAM) using 16-point (16-QAM), 64-point (64-QAM), or 256-point (256-QAM) constellations, and pulse-amplitude modulation (PAM) with a large number of levels, such as PAM-16 or PAM-32. Conversely, when many crossings or high loss are expected, an encoding scheme using lowercase alphabets can be used. Examples of such encoding schemes include binary phase shift keying (BPSK) or PAM with few levels, such as PAM-2 or PAM-4. In some embodiments, the bandwidth of the optical path may be set according to changing communication patterns or the number of photonic module boundaries where the signal is expected to cross.

[0101] The clocking of an electronic switching network can be performed using a single distributed clock. To achieve extremely low jitter at each receiver, the clock phase may be adjusted in the photonic domain. This eliminates the need for an embedded clock, which may obviate the need to encode data. By not encoding data, bandwidth expansion, latency reduction, and power reduction are possible.

[0102] VII. COMPUTING SYSTEMS BASED ON PHOTONIC COMMUNICATION PLATFORMS Computing systems can be formed that utilize the photonic communication platforms described herein. Unlike electronic distribution networks, these optical communication platforms can provide multiple copies of the same message simultaneously to multiple locations without incurring problems caused by parasitic impedance. This property enables optical distribution networks to form multicast and / or broadcast communication schemes. The optical distribution network can be dynamically reconfigured to route messages to a single node or multiple nodes. Leveraging the ability to perform broadcast and / or multicast communication, some embodiments enable the execution of MapReduce operations directly using the optical communication platform.

[0103] These types of computing systems may be used in a variety of applications, including, for example, but not limited to, high-performance computing, neural networks, machine learning networks, and deep learning networks, graphics rendering, large-scale visualization, gaming, high-frequency trading, and video streaming.

[0104] FIG. 8A illustrates an example of a computing system 800. This computing system is formed on a 4x4 photonic substrate. Of course, other dimensions and topologies are also possible. In this example, there are four processor dies 704 positioned at the center of the photonic substrate and twelve memory dies 702 surrounding these processor dies. Each die is attached to a photonic module. Each die communicates with its respective photonic module using, for example, an out-of-plane coupler (such as that illustrated in FIGS. 3B or 3C). Other embodiments may include a different number of processors and a different number of memories and / or may include other types of dies including, for example, analog accelerators, photonic accelerators, photonic memories, networking chips, and the like. In some embodiments, the computing system 800 may be provided on an interposer and communicate with the interposer using through-silicon vias.

[0105] The inset at the bottom of FIG. 8A shows the photonic module 722 in more detail. The photonic module 722 may be implemented using (or include) any of the photonic modules described above. For example, the photonic module 722 includes an optical distribution network 104. The photonic module 722 may further include waveguides (not shown in FIG. 8A) that enable optical communication with neighboring photonic modules (see, for example, waveguides 111-114 in FIG. 3A). The photonic module 722 further includes at least one fiber coupler 710, which may include edge couplers and / or out-of-plane couplers. Since the fiber coupler 710 can be coupled to the end of an optical fiber, communication with other systems is enabled. Edge couplers enable optical coupling in the plane of the photonic module. Examples of edge couplers include tapered waveguides, V-grooves, and U-grooves. In some embodiments, the edge coupler simply includes the end of a waveguide at the edge of the photonic substrate. In contrast, out-of-plane couplers (e.g., grating couplers and prisms) enable optical coupling outside the plane of the photonic chip. This particular computing system is arranged such that the bottom photonic module (as indicated by the label "fiber channel") is connected to an optical fiber. The photonic module 722 may further include one or more out-of-plane couplers (not shown in FIG. 8A) that enable optical communication with dies attached to the photonic module (see, for example, out-of-plane coupler 105 in FIG. 3A).

[0106] The upper inset in FIG. 8A shows the memory die 702 in more detail. The memory die 702 includes a memory block 720, which includes a plurality of memory units (such as solid state memories such as NAND, DRAM, SRAM, HBM, etc.). The memory die 702 further includes a communication block 724, which may include optical components for communicating with the photonic module to which the memory die is attached. For example, the communication block 724 may include an out-of-plane coupler that couples to the out-of-plane coupler of the photonic module. The memory die 702 further includes a serializer / deserializer (SERDES) block 722. The SERDES block 722 converts data from serial to parallel and vice versa. In this particular implementation, the SERDES block is positioned near the outer edge of the memory die 702, and the memory block 720 is positioned in the central part of the memory die. Of course, other arrangements are possible. FIG. 8A does not show the processor die 704 in detail, but the processor die may also include an out-of-plane coupler that couples to the out-of-plane coupler of each photonic module.

[0107] The computing system described with respect to FIG. 8A may be used as a stand-alone computing system or in combination with other computing systems. A combination of computing systems, as used herein, refers to a multi-node computing system. FIG. 8B illustrates an example of a multi-node computing system that includes four computing systems 800. Other multi-node computing systems may include, for example, dozens, or hundreds, or more units of a number of computing systems. Optical fibers 712 are used to put the computing systems in communication with each other. Each end of the optical fiber is coupled to a fiber coupler 710 of the computing system 800. In this example, the computing systems of the multi-node computing system share the same layout (same number of photonic modules, processor dies, and memory dies). However, not all embodiments are limited in this regard. In some embodiments, communication between a number of multi-node computing systems can be performed electronically by using a silicon interposer. This communication strategy may consume a reasonably small amount of power when the computing systems are placed adjacent to each other or at a distance within a few centimeters of each other in order to reduce capacitance and other parasitic resistances.

[0108] The inventors further understand that an optical module of the type described herein may serve as a transceiver that enables communication from one computing system to another or from a computer system to a host. This transceiver may be optical or electronic. Examples of optical interfaces include inter-board communication via optical fibers or inter-rack communication that utilizes a higher-level protocol such as Ethernet® or Infiniband™. Electronic interfaces to the host system include SERDES-based standards such as PCI Express. An external I / O module may manage communication between the host and the local system. This includes a direct memory access offload function for high-speed data movement between remote and local memory. The external I / O module may also provide a local interface for management, calibration, boot, and reliability and serviceability (RAS).

[0109] An optical communication platform of the type described in this specification can provide a layered network stack. An example of the stack is configured as follows. The physical layer includes an optical interconnection including an optical-electric converter, an electric-optical converter, and an optical distribution network. The data link layer includes an electronic switching network that enables connections between modules. In some embodiments, communication between nodes of the network may begin with a request message on the electronic switching network. The data link layer processes this request and performs arbitration through a mesh of electrical switches to establish a connection. At each photonic module boundary, if the request wins the arbitration, an optical link is formed. When the connection is fully established between the source and the destination, an acknowledgment can be sent back to the requester on the electronic switching network, and data transmission proceeds on the photonic module. The transport layer is involved in packetization, data integrity, and buffer allocation. The transport layer uses packetization to implement upper layer protocols on the data link layer. Flow control may implement buffer credits. If further protection from errors in the physical layer is required, data integrity including cyclic redundancy check (CRC) (e.g., retransmission), and / or forward error correction (FEC) schemes can be used.

[0110] Although some aspects and embodiments of the technology of this application have been described as above, it should be understood that those skilled in the art will readily come up with various changes, modifications, and improvements. Such changes, modifications, and improvements are intended to be within the spirit and scope of the technology described in this application. Therefore, the foregoing embodiments are presented by way of example only, and it should be understood that within the scope of the appended claims and their equivalents, embodiments of the present invention can be implemented in ways other than those specifically described. In addition, any combination of two or more features, systems, articles, materials, and / or methods described in this specification is included within the scope of this disclosure if such features, systems, articles, materials, and / or methods do not conflict with each other.

[0111] Similarly, as described, some aspects can be embodied as one or more methods. The acts performed as part of a method can be ordered in any suitable manner. Accordingly, embodiments can be configured in which the acts are performed in an order different from that illustrated, which can include performing some acts simultaneously, even if the illustrated embodiments show the acts as sequential.

[0112] It should be understood that all definitions defined and used in this specification are directed to dictionary definitions, definitions in documents incorporated by reference, and / or the ordinary meaning of the defined terms. As used in the specification and claims of this application, the indefinite articles "a" and "an" are to be understood to mean "at least one" unless clearly indicated otherwise.

[0113] As used in the specification and claims of this application, the phrase "and / or" means "either or both" of the combined elements, i.e., elements that may exist conjunctively or disjunctively.

[0114] As used in the specification and claims of this specification, the phrase "at least one" when referring to the recitation of one or more elements means at least one element selected from any one or more of the elements in the recitation of elements, but does not necessarily include at least one of every element specifically recited in the recitation of elements, and is not intended to exclude any combination of elements in the recitation of elements. It should be understood that this definition also allows for the possibility that elements other than those specifically identified within the recitation of elements referred to by the phrase "at least one" may optionally exist, whether or not they are related to those specifically identified elements.

[0115] The terms "about" and "approximately" can be used to mean within ±20% of the target value in some embodiments, within ±10% of the target value in some embodiments, within ±5% of the target value in some embodiments, and further, within ±2% of the target value in some embodiments. The terms "about" and "approximately" may include the target value.

Claims

1. A photonic system comprising: a plurality of photonic modules including at least a first photonic module and a second photonic module, each photonic module of the plurality of photonic modules being an individual instance of a photolithography shot performed using at least one common photomask; wherein the first photonic module and the second photonic module each: a first boundary line and a second boundary line defined by a boundary line of the at least one common photomask; a controllable optical distribution network; a first optical waveguide optically coupling the optical distribution network to a first neighboring photonic module among the plurality of photonic modules, the first neighboring photonic module being adjacent to the first boundary line; a second optical waveguide optically coupling the optical distribution network to a second neighboring photonic module among the plurality of photonic modules, the second neighboring photonic module being adjacent to the second boundary line; and the controllable optical distribution network is configured to selectively couple the first optical waveguide to the second optical waveguide.

2. The photonic system according to claim 1, wherein the first boundary line and the second boundary line are in opposite positions to each other.

3. The photonic system according to claim 1, wherein the first photonic module and the second photonic module each further include an out-of-plane optical coupler optically coupled to the optical distribution network.

4. In the photonic system according to claim 1, wherein the first photonic module and the second photonic module each: a third boundary line and a fourth boundary line, wherein the first boundary line and the second boundary line are in opposite positions to each other, and the third boundary line and the fourth boundary line are in opposite positions to each other; a third optical waveguide optically coupling the optical distribution network to a third neighboring photonic module among the plurality of photonic modules, the third neighboring photonic module being adjacent to the third boundary line; A fourth neighboring photonic module among the plurality of photonic modules, the fourth neighboring photonic module adjacent to the fourth boundary line, and a fourth optical waveguide optically coupling the optical distribution network to the fourth neighboring photonic module A photonic system further comprising. **Claim 5** The photonic system according to claim 4, wherein the optical distribution network is configured to be selectively arranged to optically communicate the first neighboring photonic module with a second neighboring photonic module or the third neighboring photonic module. **Claim 6** The photonic system according to claim 1, wherein the optical distribution network includes a plurality of optical switches. **Claim 7** The photonic system according to claim 1, wherein the first photonic module and the second photonic module are adjacent to each other such that the second photonic module is the first neighboring photonic module of the first photonic module. **Claim 8** A method for manufacturing a semiconductor wafer, At least some of the plurality of photonic modules are patterned on the semiconductor wafer as individual instances of photolithography shots performed using at least one common photomask such that at least some of the plurality of photonic modules are copies of template photonic modules. Patterning at least some of the plurality of photonic modules respectively includes Patterning a controllable optical distribution network; Patterning a first optical waveguide optically coupling the optical distribution network to a first neighboring photonic module among the plurality of photonic modules, the first neighboring photonic module adjacent to a first boundary line of the photonic module; Patterning a second optical waveguide optically coupling the optical distribution network to a second neighboring photonic module among the plurality of photonic modules, the second neighboring photonic module adjacent to a second boundary line of the photonic module Comprising The method in which the controllable optical distribution network is configured to selectively couple the first optical waveguide to the second optical waveguide. **Claim 9** The method according to claim 8 further includes dicing the semiconductor wafer to obtain a photonic substrate, the photonic substrate including a first photonic module among the plurality of photonic modules, a first neighboring photonic module adjacent to the first boundary line of the first photonic module, and a second neighboring photonic module adjacent to the second boundary line of the first photonic module. **Claim 10** The method according to claim 9, wherein the first boundary line and the second boundary line of the first photonic module are in opposite positions to each other. **Claim 11** In the method according to claim 8, patterning at least some of the plurality of photonic modules respectively includes patterning a third optical waveguide that optically couples the optical distribution network to a third neighboring photonic module among the plurality of photonic modules, the third neighboring photonic module being adjacent to a third boundary line of the photonic module, and further includes patterning a fourth optical waveguide that optically couples the optical distribution network to a fourth neighboring photonic module among the plurality of photonic modules, the fourth neighboring photonic module being adjacent to a fourth boundary line of the photonic module, wherein the first boundary line and the second boundary line are in opposite positions to each other, and the third boundary line and the fourth boundary line are in opposite positions to each other. **Claim 12** In the method according to claim 8, patterning at least some of the plurality of photonic modules respectively includes patterning a first photonic module using a first photolithography shot with respect to at least one common photomask, and subsequent to the first photolithography shot, patterning a second photonic module using a second photolithography shot with respect to the at least one common photomask. The method further includes. **Claim 13** A computing system comprising: A photonic substrate patterned with a plurality of photonic modules including at least a first photonic module and a second photonic module, wherein each of the first and second photonic modules is an individual instance of a photolithography shot performed using at least one common photomask, the first photonic module is optically coupled to the second photonic module, and the first and second photonic modules each:[[]] A first boundary line and a second boundary line defined by a boundary line of the at least one common photomask; A controllable optical distribution network; A first optical waveguide optically coupling the optical distribution network to a first neighboring photonic module among the plurality of photonic modules, the first neighboring photonic module adjacent to the first boundary line; A second optical waveguide optically coupling the optical distribution network to a second neighboring photonic module among the plurality of photonic modules, the second neighboring photonic module adjacent to the second boundary line; Including; The controllable optical distribution network is configured to selectively couple the first optical waveguide to the second optical waveguide; A photonic substrate; A first die communicating with the first photonic module; A second die communicating with the second photonic module; A computing system including.

14. The computing system according to claim 13, wherein the first die includes a processor and the second die includes a memory.

15. The computing system according to claim 13, further including a laser die coupled to the photonic substrate.

16. The computing system according to claim 13, wherein the first boundary line and the second boundary line are in opposite positions.

17. In the computing system according to claim 13, The first photonic module and the second photonic module each include an out-of-plane optical coupler; The first die is optically coupled to the out-of-plane optical coupler of the first photonic module; The computing system, wherein the second die is optically coupled to the out-of-plane optical coupler of the second photonic module. **Claim 18** The computing system according to claim 13, wherein the first die is coupled to a first side of the photonic substrate, and the second die is coupled to a second side of the photonic substrate that is located opposite to the first side. **Claim 19** The computing system according to claim 13, further comprising a third die stacked on top of the first die. **Claim 20** The computing system according to claim 13, wherein the first photonic module and the second photonic module share a boundary line such that the first photonic module is adjacent to the second photonic module. **Claim 21** In the computing system according to claim 13, the first die is mounted above or below the first photonic module, and the second die is mounted above or below the second photonic module. **Claim 22** In the computing system according to claim 13, the first die communicates electronically with the first photonic module, and the second die communicates electronically with the second photonic module.

Images