Multi-stack computing chip and memory architecture

Abstract

A multi-stack compute chip and memory architecture is described. According to the described technology, a package includes multiple computing stacks, each computing stack including at least one compute chip and memory. The package also includes one or more interconnects coupling the computing stacks to at least one other computing stack for sharing memory in a coherent manner across the multiple computing stacks.

Description

[Technical Field]

[0001] (Related Applications) This application claims priority under 35 U.S.C. §119(e) to U.S. Provisional Patent Application No. 63 / 484,183, entitled "Multi-Stack Compute Chip and Memory Architecture," filed February 9, 2023, the entire disclosure of which is incorporated herein by reference, and to U.S. Non-Provisional Patent Application No. 18 / 390,893, entitled "Multi-Stack Compute Chip and Memory Architecture," filed December 20, 2023. [Background technology]

[0002] Computing technology is constantly advancing, as evidenced, for example, by the proliferation of machine learning into many different fields. As computing applications advance, computer hardware also advances, providing improved processing and memory, for example, faster speeds, higher performance, lower power consumption, etc. [Brief explanation of the drawings]

[0003] [Figure 1] FIG. 1 is a block diagram of a non-limiting exemplary system having multiple stacks of computing chips and memory interconnected to form a package, according to some embodiments. [Figure 2] FIG. 1 is a block diagram of a non-limiting example of a multi-stack package having stacks arranged in a grid. [Figure 3] FIG. 1 is a block diagram of a non-limiting example of a multi-stack package having stacks arranged in an array. [Figure 4] FIG. 10 is a block diagram of a non-limiting example in which a multi-stack package is expanded through connection with additional multi-stack packages. [Figure 5]1A-1C illustrate non-limiting examples of different topologies for connecting stacks of multi-stack packages with interconnects. [Figure 6] FIG. 2 illustrates a non-limiting example of a configuration of a stack with compute chips and memory, according to some embodiments. [Figure 7] FIG. 1 illustrates a procedure in an exemplary embodiment of a multi-stack compute chip and memory architecture. [Figure 8] 1A-1C illustrate a procedure in an exemplary embodiment for manufacturing a multi-stack package. DETAILED DESCRIPTION OF THE INVENTION

[0004] (overview) A multi-stack compute chip and memory architecture is described. According to the described technology, a package includes multiple computing stacks, and in some variations, each computing stack includes at least one compute chip and memory (e.g., at least one memory die). By way of example, the memory is stacked memory, such as a stack of dynamic random access memory (DRAM), stacked three-dimensionally (3D) above (or below) the compute chips to form a stack. This allows the compute chips high-bandwidth access to data in the memory of each stack in a power-efficient manner. The package also includes one or more interconnects coupling the computing stacks to at least one other computing stack for coherent memory sharing across the multiple computing stacks. Through the interconnects and memory of the multiple stacks, the package provides such shared, coherent memory with non-uniform memory access (NUMA) characteristics.

[0005] In some aspects, the techniques described herein relate to an apparatus that includes a plurality of computing stacks, wherein a first computing stack and a second computing stack of the plurality of computing stacks each include at least one compute chip and a memory, and one or more interconnects that couple the first computing stack to at least the second computing stack for sharing the memory.

[0006] In some aspects, the technology described herein relates to an apparatus in which the memory is one or more memory dies.

[0007] In some aspects, the technology described herein relates to an apparatus in which the memory of a first computing stack includes dynamic random access memory (DRAM) and the memory of a second computing stack includes non-volatile memory.

[0008] In some aspects, the technology described herein relates to an apparatus in which the memory of the first computing stack includes dynamic random access memory (DRAM) and non-volatile memory.

[0009] In some aspects, the techniques described herein relate to an apparatus, in which a memory of a first computing stack includes a first portion and a second portion, the first portion being embedded in at least one compute chip, and the second portion being separate from and communicatively coupled to the at least one compute chip.

[0010] In some aspects, the technology described herein relates to an apparatus in which at least one computing chip includes at least one central processing unit, graphics processing unit, field programmable gate array, accelerator, or digital signal processor.

[0011] In some aspects, the techniques described herein relate to an apparatus in which a first computing stack further includes one or more memory request units configured to perform at least one of: sending and receiving data over one or more interconnects between a memory of the first computing stack and a second computing stack; providing coherent shared memory across the memories of the multiple computing stacks; or providing coherent shared memory across the memories of a subset of the multiple computing stacks.

[0012] In some aspects, the techniques described herein relate to an apparatus in which at least one compute chip of a first computing stack is configured to access data from a memory of the first computing stack faster than it accesses data from a memory of a second computing stack, and in which the at least one compute chip of the first computing stack accesses data from the memory of the second computing stack via one or more interconnects.

[0013] In some aspects, the technology described herein relates to a device in which one or more interconnects are disposed on or within at least one of a silicon interposer, a silicon bridge, a glass interposer, an organic package, or a silicon photonic interconnect.

[0014] In some aspects, the technology described herein relates to an apparatus in which multiple computing stacks are interconnected with one or more interconnects in an array topology.

[0015] In some aspects, the technology described herein relates to an apparatus in which multiple computing stacks are interconnected with one or more interconnects in a grid topology.

[0016] In some aspects, the technology described herein relates to an apparatus that is a multi-stack package communicatively coupled to at least one additional multi-stack package.

[0017] In some aspects, the technology described herein relates to a device in which the memory of a first computing stack is disposed in a stacked configuration above or below at least one compute chip of the first computing stack, the stacked configuration being disposed on a substrate of the device.

[0018] In some aspects, the technology described herein relates to an apparatus in which a memory of a first computing stack is arranged in a parallel configuration with at least one compute chip of the first computing stack.

[0019] In some aspects, the technology described herein relates to a device in which the parallel configuration is disposed on a substrate of the device.

[0020] In some aspects, the technology described herein relates to a device in which the parallel configuration is disposed in a stacked configuration above or below a circuit die of a first computing stack, and the stacked configuration is disposed on a substrate of the device.

[0021] In some aspects, the technology described herein relates to an apparatus in which the circuit die includes at least one of a memory controller, a cache, a data fabric, a network on a chip (NoC), or a memory interface circuit.

[0022] In some aspects, the technology described herein relates to a system on package (SoP) that includes a plurality of computing stacks, each computing stack of the plurality of computing stacks including at least one compute chip and memory, and at least a first computing stack of the plurality of computing stacks coupled to at least a second computing stack and a third computing stack of the plurality of computing stacks via an interconnect to share the memory, and one or more interfaces for coupling the system on package to at least one external device.

[0023] In some aspects, the technology described herein relates to a system-on-package (SoP) in which the at least one external device includes at least one of an integrated circuit, an external memory, a motherboard, or an additional system-on-package.

[0024] In some aspects, the techniques described herein relate to a method for manufacturing a multi-stack package, the method including: forming a plurality of computing stacks, wherein a first computing stack and a second computing stack of the plurality of computing stacks each include at least one computational chip and a memory; and arranging the plurality of computing stacks on a substrate, wherein the first computing stack and the second computing stack are electrically connected on the substrate via one or more interconnects to share the memory.

[0025] 1 is a block diagram of a non-limiting exemplary system 100 having multiple stacks of computing chips and memory interconnected to form a package, according to some embodiments. In particular, the illustrated example shows two views of system 100, including a top-down "plan" view and a side-sectional view. It should be understood that the components are not drawn to scale in either view, and that in implementations, their size and positioning relative to one another may vary. Furthermore, the number of components of system 100 may vary between variations without departing from the spirit or scope of the described technology.

[0026] In accordance with the described technology, the system 100 is or includes a multi-stack package 102 having multiple stacks 104, such as at least a first stack 104 and a second stack 104. The figures include an ellipsis to indicate that in one or more embodiments, the multi-stack package 102 includes three or more stacks 104, some examples of which are shown in Figures 2-5.

[0027] The multiple stacks 104 include a compute chip 106 and a memory 108. For example, each stack 104 includes at least a compute chip 106 and a memory 108, such as a memory die. In variations, a stack 104 includes two or more compute chips 106 (e.g., two or more compute chips) and / or more memory than a single memory die (e.g., multiple memory dies and / or at least one memory die and additional memory embedded in a compute chip 106). By way of example, a stack 104 includes one or more compute chips 106 and stacked memory 108. In one or more embodiments, the memory 108 is stacked directly above one or more compute chips 106. In one or more embodiments, the memory 108 is stacked directly below one or more compute chips 106. In one or more embodiments, the compute chip 106 is disposed between a first portion of the memory 108 and a second portion of the memory 108, with one portion stacked directly above the one or more compute chips 106 and the other portion stacked directly below the one or more compute chips 106. Alternatively or additionally, the memory 108 and the compute chip 106 are stacked alternately. It should be understood that in variations, the one or more compute chips 106 and the memory 108 (e.g., one or more memory dies) may be arranged in a different manner to form the stack 104 without departing from the described techniques.

[0028] The compute chips 106 and memories 108 of a particular stack 104 are coupled to one another using any one or more of a variety of wired or wireless connection types. Exemplary wired connections include, but are not limited to, one or more memory channels, buses (e.g., data buses), interconnects, through-silicon vias, data links (e.g., 1024 data links), traces, photonic interconnects, and planes, to name a few. The stacked configuration of compute chips 106 and memories 108 allows the compute chips 106 to have high-bandwidth access to data in the memories 108 in their respective stacks 104, and enables this access with reduced power consumption. This is due to shorter data communication paths compared to configurations in which the memories 108 are physically and / or topologically further away from the compute chips 106, as well as shorter data communication paths compared to memories 108 in another stack 104.

[0029] The depicted example also shows interconnects 110 that couple (e.g., communicatively couple) the stacks 104 of the multi-stack package 102. According to the described technology, the stacks 104 are connected to at least one other stack 104 by one or more interconnects 110. Through these interconnects 110, the system 100 transfers data between the stacks 104. The interconnects 110 also enable other stacks 104 to access data maintained in their own memories 108. In at least one scenario, for example, one or more interconnects 110 enable the compute chips 106 of a first stack 104 to access data loaded into the memories 108 of at least a second stack 104. Based on this, the multi-stack package 102 is configured to provide a coherent shared memory using the memories 108 of the multiple stacks 104. For example, the multi-stack package 102 may use the memory 108 of a subset of the stacks (e.g., fewer than all of the stacks) to provide such coherent shared memory. Alternatively, the multi-stack package 102 may use at least a portion of the memory 108 of all of the stacks 104 to provide such coherent shared memory.

[0030] In one or more embodiments, the compute chips 106 are electronic circuits that perform various operations on and / or using data in memory 108, such as data in memory 108 of the stack 104 of the compute chip 106 or data in memory 108 of at least one other stack 104. By way of example and not limitation, such operations are associated with programs, applications, and / or threads (not shown). In accordance with the described technology, the compute chips 106 of a given stack may be any one or more of a variety of processing units, such as a central processing unit (CPU), a graphics processing unit (GPU), an accelerator, an accelerated processing unit (APU), a parallel accelerated processor, a digital signal processor, an artificial intelligence (AI) or machine learning accelerator, a field programmable gate array (FPGA), etc. In variations, the compute chips 106 may correspond to one or more different types of components, such as a cache, without departing from the spirit or scope of the described technology.

[0031] While a single compute chip 106 is illustrated in each stack 104 of the multi-stack package 102, the stacks 104 optionally include any number of compute chips 106 of the same or different types. In one or more embodiments, each stack 104 of the multi-stack package 102 includes one or more compute chips 106 of the same type, e.g., each stack includes the same processing unit (subject to manufacturing variations) and / or the same combination of processing units (subject to manufacturing variations). However, in other embodiments, at least one stack 104 has one or more compute chips 106 that are different from at least one other stack 104, e.g., the compute chip 106 of a first stack 104 is a CPU and the compute chip 106 of a second stack 104 is a different type of CPU or is a GPU.

[0032] Memory 108 is a device or system used to store information, such as for immediate use within the device by the computing chips 106 of a respective stack 104 or by the computing chips 106 of at least one other stack 104. In one or more embodiments, memory 108 corresponds to semiconductor memory in which data is stored in memory cells on one or more integrated circuits. In at least one example, memory 108 corresponds to or includes volatile memory, such as random-access memory (RAM), dynamic random-access memory (DRAM), synchronous dynamic random-access memory (SDRAM), phase-change memory (PCM), memristors, static random-access memory (SRAM), etc. In variations, memory 108 is packaged or configured in any of a variety of different ways.

[0033] Further example memory configurations include low-power double data rate (LPDDR), also known as LPDDR SDRAM, which is a type of synchronous dynamic random access memory. In variations, LPDDR consumes less power than other types of memory and / or has a form factor suitable for devices such as mobile computers and mobile phones. Examples of LPDDR include, but are not limited to, low-power double data rate 2 (LPDDR2), low-power double data rate 3 (LPDDR3), low-power double data rate 4 (LPDDR4), and low-power double data rate 5 (LPDDR5).

[0034] In at least one variation, memory 108 is stacked memory, one example of which is stacked DRAM. Alternatively or additionally, memory 108 corresponds to or includes non-volatile memory, examples of which include ferroelectric RAM, magnetoresistive RAM, flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), and electronically erasable programmable read-only memory (EEPROM). It should be understood that memory 108 can be configured in various manners without departing from the spirit or scope of the described technology.

[0035] In one or more variations, at least one stack 104 of the multi-stack package 102 includes a cache (or two or more caches) in addition to or instead of the memory 108. Furthermore, while described throughout as having one or more compute chips 106 and memory 108 (e.g., one or more memory dies), in variations, the stacks 104 of the multi-stack package 102 include additional and / or different components stacked together in accordance with the described techniques.

[0036] Although not shown, in one or more embodiments, the memory controller manages the compute chip 106's access to the memory 108, such as by sending read and write requests to the memory 108 and receiving responses from the memory 108.

[0037] As described above, the multi-stack package 102 includes one or more interconnects 110 connecting the stacks 104. For example, the interconnects 110 are configured to connect at least two stacks 104 and route data between the at least two stacks 104. In other words, the interconnects 110 enable data transfer and / or exchange between the at least two stacks. Broadly speaking, the interconnects 110 are components, systems, and / or devices through which data can be transferred between endpoints, e.g., between the at least two stacks 104. In variations, the multi-stack package 102 includes multiple interconnects 110. In one or more variations, the interconnects 110 are implemented on or at least partially within a silicon interposer (e.g., a passive silicon interposer or an active silicon interposer), a glass interposer, a silicon bridge, an organic package, or a photonic interconnect (e.g., a silicon photonic interconnect), to name just a few. Alternatively or additionally, interconnect 110 is implemented as a bus (e.g., a data bus), data links (e.g., 1024 data links), traces, and / or planes. In variations, interconnect 110 is configured differently without departing from the spirit or scope of the described technology.

[0038] According to the described techniques, the interconnect 110 connects the stacks 104 of the multi-stack packages 102 in any of a variety of topologies, such as a two-dimensional, or 2D, grid, to facilitate data communication between the stacks 104. Using the interconnect 110, the system 100 can implement coherent shared memory across the individual memories 108 of the multiple stacks 104 of the multi-stack packages 102. Exemplary topologies with different dimensions are described in more detail below in connection with Figures 2-5.

[0039] The depicted example also shows a memory request unit 112. In one or more embodiments, the memory request unit 112 is a logical block configured to manage memory 108, for example, for an individual stack 104, for more than one stack 104, and / or in cooperation with at least one additional memory request unit 112 (e.g., for another stack 104). In one or more embodiments, one or more of the stacks 104 includes multiple memory request units, each performing different operations to implement the techniques described above and below. In one variation, for example, at least one of the stacks 104 includes a first memory request unit that transmits and / or receives data via the interconnect 110 and also includes a second memory request unit configured to provide a coherent shared memory.

[0040] In the illustrated example, each of the stacks 104 is shown with a respective memory request unit 112. However, in one or more variations, each stack 104 includes two or more memory request units, as just described, for example, to perform a dedicated subset of memory-based operations. In at least one variation, the memory request unit 112 manages the memory 108 of two or more stacks 104, such that at least one of the stacks 104 does not include a memory request unit 112. For example, one memory request unit 112 manages the memory 108 of a subset of the stacks 104 (but at least two stacks 104). In another example, one memory request unit 112 manages the memory 108 of all stacks 104 in the multi-stack package 102. It should be understood that the number of memory request units 112 implemented in the multi-stack package 102 varies among variations.

[0041] In addition to implementing different numbers of memory request units 112 in the multi-stack package 102, in variations, the memory request units 112 are implemented in different components of the multi-stack package 102. In at least one variation, for example, the memory request units 112 are implemented in the compute chip 106, as in the illustrated example. However, the memory request units 112 can be implemented in other components in variations. For example, the memory request units 112 are configured as dedicated circuits integrated (e.g., soldered) with each stack (but separate, for example, from the compute chip 106) that perform the various operations described above and below. Alternatively or additionally, the memory request units 112 are configured as microcontrollers, such as a microcontroller located on a die integrated with each stack or located on a die of the multi-stack package 102, that execute firmware to perform the various operations described above and below. In one or more embodiments, the memory request units 112 are shared among multiple stacks 104. The memory request unit 112 may be implemented in one or more of a variety of components in accordance with the described techniques.

[0042] In one or more embodiments, one or more memory request units 112 cause the memories 108 of the multiple stacks 104 of the multi-stack package 102 (or the memories 108 of at least a subset of the stacks 104) to operate as a shared, coherent memory. These memory request units 112 do so, for example, according to one or more memory management techniques. For example, one or more of the memory request units 112 expose the memory 108 across the multiple stacks 104 as a coherent shared memory with non-uniform memory access (NUMA) properties. Such memory has NUMA properties because data in the memory 108 of the same stack 104 as a given compute chip 106 is accessible from the location of the given compute chip 106 more quickly than data in the memory 108 of a different stack (e.g., across at least one interconnect 110). This is due, at least in part, to differences in physical and topological distances and the number of interfaces and components through which data in other stacks 104 (e.g., remote stacks) is accessed.

[0043] Additionally or alternatively, one or more of the memory request units 112 are configured to cause high-speed data movement between one or more of the stacks 104, e.g., a subset of the stacks. In at least one variation, one or more of the memory request units 112 implement individual data movers. In at least one variation, these data movers are usable to implement a message passing interface (MPI) with individual ranks for one or more of the stacks 104, e.g., a subset of the stacks. In one or more embodiments, the data movers use one or more other formats and / or standards different from MPI.

[0044] The multi-stack package 102 optionally includes one or more additional controllers for linking to additional devices, such as a Peripheral Component Interconnect Express (PCIe) controller, a Serial Advanced Technology Attachment (SATA) controller, a Universal Serial Bus (USB) controller, a Serial Peripheral Interface (SPI) controller, a Low Pin Count (LPC) controller, HyperTransport (HT), Compute eXpress Link (CXL), etc. In variations, the multi-stack package 102 includes one or more additional components not shown in this example, such as interfaces (e.g., for connecting with components and / or systems external to the multi-stack package 102), controllers, system managers, and optical components, to name just a few. By way of example, the multi-stack package 102 may be configured to connect to and communicate with at least one other multi-stack package 102 or a different component or system using one or more such additional components, such as one or more silicon photonic interconnects and / or network interface cards (NICs).

[0045] System 100 is configured for incorporation into a device or apparatus. By way of example and not limitation, examples of different types of devices or apparatuses into which system 100 may be incorporated include a server, a personal computer (e.g., a desktop or tower computer), a smartphone or other wireless phone, a tablet or phablet computer, a notebook computer, a laptop computer, a wearable device (e.g., a smartwatch, an augmented reality headset or device, a virtual reality headset or device), an entertainment device (e.g., a game console, a portable gaming device, a streaming media player, a digital video recorder, a music or other audio playback device, a television, a set-top box), an Internet of Things (IoT) device, an automobile computer, a computer in another type of vehicle (e.g., a scooter, an e-bike, a motorcycle), a system on chip (SoC), a system on package (SoP), sometimes referred to as a system in a package (SiP), and the like.

[0046] FIG. 2 is a block diagram of a non-limiting example 200 of a multi-stack package having stacks arranged in a grid.

[0047] In the illustrated example 200, the multi-stack package 102 includes multiple stacks 104, which can be configured in various manners as described above. The stacks 104 are connected to other stacks 104 in the multi-stack package 102 in a grid, e.g., a two-dimensional (2D) grid. As described above, interconnects 110 connect the stacks 104 in the multi-stack package 102. Also, as described above, the interconnects 110 allow the memory 108 of an individual stack 104 to be shared with one or more additional stacks 104. In this manner, the compute chips 106 in an individual stack 104 can access data in the memory 108 of the same stack 104 as well as data in the memory 108 of one or more additional stacks 104 in the multi-stack package 102 via the interconnects 110.

[0048] In this example, the multi-stack package 102 is shown to include interface components 202. These interface components 202 support interaction with different devices and / or systems, such as additional multi-stack packages 102, other integrated circuits, external memory, a motherboard, etc. Examples of interface components 202 include, but are not limited to, a network interface card (NIC), a photonic interconnect component, one or more sockets, a Peripheral Component Interconnect Express (PCIe) component, a Serial Advanced Technology Attachment (SATA) component, a Universal Serial Bus (USB) component, a Serial Peripheral Interface (SPI) component, a Low Pin Count (LPC) component, a Compute eXpress Link (CXL) component, etc. In one or more embodiments, the interconnect 110 connects the stacks 104 to the interface components 202. However, in at least one variation, the stacks 104 are connected to the interface components 202 (or other components) using couplings different from those used to connect the stacks 104 from one stack to another.

[0049] In one or more embodiments, a system or device includes additional and / or different types of memory external to the multi-stack package 102. Alternatively or additionally, a system or device includes additional and / or different types of memory within one or more of the multi-stack packages 102, where the additional and / or different types of memory are separate from the memory 108 in the stack 104. For example, one or more additional memories are integrated within the package and are not part of the stack 104. Instead, the one or more additional memories are located “outside” the interface components 202 relative to the multi-stack package 102, such as on the opposite side of at least one interface component 202 from the multi-stack package 102. With respect to example 200, for example, from the illustrated top-down perspective, one or more of the additional memories may be located to the left of the left-most interface component 202, above the top-most interface component 202, to the right of the right-most interface component 202, and / or below the bottom-most interface component 202. For different configurations of the stack 104 of multi-stack packages 102, see FIG. 3 .

[0050] FIG. 3 is a block diagram of a non-limiting example 300 of a multi-stack package having stacks arranged in an array.

[0051] In the illustrated example 300, the multi-stack package 102 includes multiple stacks 104, which can be configured in various manners as described above. The stacks 104 are connected to other stacks 104 in the multi-stack package 102 in an array, e.g., a one-dimensional (1D) array. As described throughout, interconnects 110 connect the stacks 104 in the multi-stack package 102. Also, as described above, the interconnects 110 allow the memory 108 of an individual stack 104 to be shared with one or more additional stacks 104. In this manner, the compute chips 106 in an individual stack 104 can access data in the memory 108 of the same stack 104, as well as data in the memory 108 of one or more additional stacks 104 in the multi-stack package 102, via the interconnects 110.

[0052] The examples shown in FIGS. 2-3 are merely illustrative, and the layout of the stacks 104 of the multi-stack package 102 may vary among variations without departing from the spirit or scope of the described technology.

[0053] FIG. 4 is a block diagram of a non-limiting example 400 in which a multi-stack package is expanded through connection with additional multi-stack packages.

[0054] The illustrated example 400 includes multiple multi-stack packages 102. The multi-stack packages 102 are connected via communicative couplings 402. This illustrates a scenario in which the multi-stack package 102 is expanded by connecting the multi-stack package 102 to one or more additional multi-stack packages 102. In one or more embodiments, this allows for shared coherent memory across multiple multi-stack packages 102 in addition to shared coherent memory across multiple stacks of individual packages.

[0055] FIG. 5 shows non-limiting examples 500 of different topologies for connecting stacks of multi-stack packages with interconnects.

[0056] In particular, the illustrated example 500 includes a first topology 502, a second topology 504, a third topology 506, and a fourth topology 508. Each of the topologies shows multiple stacks 104 and interconnects 110 connecting the stacks. The first topology 502 is an array of stacks 104 and is an example of a one-dimensional, or 1D, topology. The second topology 504 is a grid of stacks 104 (e.g., four stacks 104) and is an example of a two-dimensional, or 2D, topology. The third topology 506 is another grid of stacks 104 (e.g., eight stacks 104) and is an example of a three-dimensional, or 3D, topology. The fourth topology 508 is another grid of stacks 104 (e.g., 16 stacks 104) and is an example of a four-dimensional, or 4D, topology. In one or more embodiments, the dimensionality of the topologies is based on the number of other computing stacks to which an individual computing stack is connected via the interconnects 110. In at least one variation, for example, the dimension is based on the number of computing stacks connected via interconnects 110, where a computing stack connected to the fewest number of other computing stacks in the topology. As an example, each computing stack 104 in the third topology 506 is connected to three other computing stacks, and thus the third topology 506 is a 3D topology. In contrast, each computing stack 104 in the fourth topology 508 is connected to four other computing stacks, and thus the fourth topology 508 is a 4D topology. In other words, the dimension of a topology relates to the number of interconnections between a given stack and other stacks, e.g., the number of other stacks to which a given stack is connected. For example, a stack 104 having interconnections 110 to N different stacks corresponds to an ND topology. As another specific example (not shown), if a given stack 104 is connected to 10 other stacks 104 via interconnects, the topology corresponds to a 10D topology.

[0057] In the alternative, it should be understood that the interconnects 110 and stacks 104 may be arranged in many different topologies, either higher or lower dimensional, without departing from the spirit or scope of the described technology.

[0058] FIG. 6 illustrates a non-limiting example 600 of a configuration of a stack with compute chips and memory, according to some embodiments.

[0059] The illustrated example 600 shows an example stack 602 having an arrangement of hardware components used in at least one variation in accordance with the described technology. By way of example, one or more components of the first stack 104 or the second stack 104 are arranged as in stack 602 in example 600. It should be understood that the components of the stacks of a multi-stack package may be arranged in different manners without departing from the spirit or scope of the described technology.

[0060] In the depicted example 600, stack 602 includes compute chip 106, memory 108, and circuit die 604. In one or more embodiments, stack 602 or one or more components of stack 602 (e.g., circuit die 604) are coupled to or integrated with substrate 606, one example of which is a system-on-chip (SOC) substrate. By way of example and not limitation, substrate 606 corresponds to multi-stack package 102.

[0061] Notably, the configuration of the illustrated example 600 differs from the configuration shown in the side view of FIG. 1 . In system 100, compute chip 106 and memory 108 are arranged in a vertically stacked configuration, e.g., memory 108 is disposed “above” compute chip 106. In contrast, in example 600, compute chip 106 and memory 108 have a side-by-side configuration, such that memory 108 is disposed alongside compute chip 106, rather than vertically above or below compute chip 106. It should be understood that in one or more embodiments, different stacks of a multi-stack package include stacks having different component configurations, such as a first stack having the configuration shown in FIG. 1 and a second stack having the configuration shown in example 600. However, in at least one variation, all stacks of a multi-stack package have the same component configuration, such as, for example, all stacks of a multi-stack package having a configuration similar to that shown in example 600.

[0062] In the depicted example 600, the circuit die includes a memory request unit 112. In one or more embodiments, the memory request unit 112 is implemented using different and / or additional components than the circuit die 604. The circuit die 604 can be configured as and / or with any of a variety of semiconductor components, including, but not limited to, a memory controller, cache, data fabric, network-on-chip (NoC), and memory interface circuitry, to name just a few. In one or more embodiments, the stack 602 includes multiple circuit dies.

[0063] While this example 600 shows a single compute chip 106 and memory 108, in one or more variations, multiple layers of compute chips 106 and memory 108 are stacked on the circuit die 604, whereby at least a second layer having compute chips 106 and memory 108 is stacked on a first layer to form a “taller” stack.

[0064] FIG. 7 illustrates a procedure 700 in an exemplary embodiment of a multi-stack compute chip and memory architecture.

[0065] Data in the first memory is accessed for use by the compute chip (block 702). According to principles described herein, the first memory and the compute chip are coupled together to form a first stack of a multi-stack package. By way of example, data in memory 108 of a first stack 104 of a multi-stack package 102 is accessed by a memory controller (not shown) of the first stack 104 for use by a compute chip 106 of the first stack 104. For example, the data is accessed for use by the compute chip 106 to execute instructions of an application, such as an operating system, machine learning-based tasks, etc.

[0066] The data in the second memory is accessed for use by the compute chips (block 704). According to principles described herein, the second memory is disposed in a second stack of a multi-stack package, the second stack being communicatively coupled to the first stack by one or more interconnects of the multi-stack package. By way of example, data in the memory 108 of the second stack 104 of the multi-stack package 102 is accessed by a memory controller (not shown) of the second stack 104 for use by the compute chips 106 of the first stack 104. In at least one variation, the data is communicated from the second stack 104 to the first stack 104 via the interconnect 110. The compute chips 106 of the first stack 104 then use (e.g., process) the data retrieved from the memory 108 of the second stack 104. For example, the compute chip 106 of the first stack 104 executes instructions (eg, of an application) on data retrieved from the memory 108 of the first stack 104 .

[0067] FIG. 8 illustrates a procedure 800 in an exemplary embodiment for manufacturing a multi-stack package.

[0068] A plurality of computing stacks are formed (block 802). In accordance with the principles herein, a first computing stack and a second computing stack of the plurality of computing stacks include at least one computing chip and a memory. By way of example, a plurality of stacks 104 are formed, the plurality of stacks including at least a first stack 104 and a second stack 104. In this example, the first stack 104 and the second stack 104 are each formed to include a computing chip 106 and a memory 108. In one or more embodiments, one or more of the stacks are formed by disposing the memory 108 above or below the computing chip 106 in a stacked configuration, an example of which is shown in FIG. 1. Alternatively or additionally, one or more of the stacks are formed by disposing the memory 108 in a parallel configuration with the computing chip 106, an example of which is shown in FIG. 6. The computing chip 106 and memory 108 of a particular stack can be electrically connected to enable communication between their components in various manners, such as using through-silicon vias or any of the various connections described above, without departing from the spirit or scope of the described technology.

[0069] The multiple computing stacks are disposed on a substrate (block 804). In accordance with the principles herein, the computing stacks are disposed on the substrate such that the first computing stack and the second computing stack are electrically connected on the substrate via one or more interconnects to share memory. By way of example, multiple computing stacks including the first computing stack 104 and the second computing stack 104 are disposed on a substrate, such as substrate 606. The multiple computing stacks are alternatively arranged in different topologies, examples of which include, but are not limited to, the array topology of FIG. 5, a 2D topology, a 3D topology, and a 4D topology. On the substrate 606, the first computing stack 104 and the second computing stack 104 are electrically connected, such as via one or more interconnects 110. Via the interconnects 110, the first computing stack 104 and the second computing stack 104 communicate to share their respective memories 108, for use by the compute dies 106 of the other stacks, e.g., having NUMA properties.

[0070] It should be understood that many variations are possible based on the disclosure herein, and although features and elements are described above in particular combinations, each feature or element can be used alone without the other features and elements, or in various combinations with or without the other features and elements.

[0071] The various functional units illustrated in the figures and / or described herein (including, where appropriate, the multi-stack package 102, the plurality of stacks 104, the compute chip 106, the memory 108, the interconnect 110, and the memory request unit 112) may be implemented in any of a variety of different ways, such as hardware circuits, software or firmware running on a programmable processor, or any combination of two or more of hardware, software, and firmware. The provided methods may be performed in any of a variety of devices, such as a general-purpose computer, a processor, or a processor core. Suitable processors include, by way of example, a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), a graphics processing unit (GPU), a parallel-accelerated processor, multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), and / or a state machine.

[0072] In one or more embodiments, the methods and procedures provided herein may be implemented in a computer program, software, or firmware embodied in a non-transitory computer-readable storage medium for execution by a general-purpose computer or processor. Examples of non-transitory computer-readable storage media include read only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs).

Claims

1. 1. An apparatus comprising: a plurality of computing stacks, wherein a first computing stack and a second computing stack of the plurality of computing stacks each include at least one compute chip and a memory; one or more interconnects coupling the first computing stack to at least the second computing stack for sharing the memory. Device.

2. the memory being one or more memory dies; 10. The apparatus of claim 1.

3. the memory of the first computing stack comprises dynamic random access memory (DRAM) and the memory of the second computing stack comprises non-volatile memory; 10. The apparatus of claim 1.

4. the memory of the first computing stack includes dynamic random access memory (DRAM) and non-volatile memory; 10. The apparatus of claim 1.

5. the memory of the first computing stack includes a first portion and a second portion, the first portion being embedded in the at least one computing chip, and the second portion being separate from and communicatively coupled to the at least one computing chip; 10. The apparatus of claim 1.

6. the at least one computing chip includes at least one central processing unit, graphics processing unit, field programmable gate array, accelerator, or digital signal processor; 10. The apparatus of claim 1.

7. The first computing stack comprises: transmitting and receiving data over the one or more interconnects between the memory of the first computing stack and the second computing stack; providing a coherent shared memory across the memories of the multiple computing stacks; or providing a coherent shared memory across memories of a subset of the plurality of computing stacks; one or more memory request units configured to perform at least one of 10. The apparatus of claim 1.

8. the at least one compute chip of the first computing stack is configured to access data in the memory of the first computing stack faster than it accesses data in the memory of the second computing stack, and the at least one compute chip of the first computing stack accesses data in the memory of the second computing stack via the one or more interconnects; 10. The apparatus of claim 1.

9. the one or more interconnects are disposed on or within at least one of a silicon interposer, a silicon bridge, a glass interposer, an organic package, or a silicon photonic interconnect; 10. The apparatus of claim 1.

10. The apparatus of claim 1 , wherein the plurality of computing stacks are interconnected with the one or more interconnects in an array topology.

11. the plurality of computing stacks are interconnected with the one or more interconnects in a grid topology; 10. The apparatus of claim 1.

12. the device is a multi-stack package communicatively coupled to at least one additional multi-stack package; 10. The apparatus of claim 1.

13. the memory of the first computing stack is disposed in a stacked configuration above or below the at least one compute chip of the first computing stack, the stacked configuration being disposed on a substrate of the device; 10. The apparatus of claim 1.

14. the memory of the first computing stack is arranged in a parallel configuration with the at least one compute chip of the first computing stack; 10. The apparatus of claim 1.

15. the parallel arrangement is disposed on a substrate of the device; 15. The apparatus of claim 14.

16. the parallel configuration is disposed in a stacked configuration above or below a circuit die of the first computing stack, the stacked configuration being disposed on a substrate of the device; 15. The apparatus of claim 14.

17. The circuit die includes: memory controller, cache, Data Fabric, Network on Chip (NoC), or a memory interface circuit; at least one of 17. The apparatus of claim 16.

18. A system on package (SoP), comprising: Multiple computing stacks and one or more interfaces; each computing stack of the plurality of computing stacks includes at least one computing chip and a memory; at least a first computing stack of the plurality of computing stacks is coupled to at least a second computing stack and a third computing stack of the plurality of computing stacks via an interconnect to share the memory; The one or more interfaces: coupling the system on package to at least one external device; System on Package (SoP).

19. The at least one external device integrated circuits, External memory, Motherboard, or Additional system-on-packages, at least one of The SoP of claim 18.

20. 1. A method for manufacturing a multi-stack package, comprising: forming a plurality of computing stacks, wherein a first computing stack and a second computing stack of the plurality of computing stacks each include at least one compute chip and a memory; disposing the plurality of computing stacks on a substrate, wherein the first computing stack and the second computing stack are electrically connected on the substrate via one or more interconnects to share the memory; method.

Images