Vertically integrated memory systems and associated systems and methods

Abstract

The invention relates to vertically integrated memory systems and associated systems and methods. A system-in-package (SiP) with a combined high bandwidth memory (HBM) device and associated systems and methods are disclosed herein. In some embodiments, the SiP includes a base substrate (e.g., a silicon interposer), a processing unit carried by the base substrate, and an HBM device carried by the base substrate. The combined HBM device may be electrically coupled to the processing unit through one or more traces. Further, the combined HBM device may include an interface die, one or more volatile memory dies (e.g., volatile main memory components) carried by the interface die, and one or more non-volatile memory dies carried by the one or more memory dies. The combined HBM device may also include a shared bus electrically coupled to the interface die, the volatile memory die, and the non-volatile memory die to establish a communication path therebetween.

Description

Technical Field

[0001] The present technology relates generally to vertically stacked semiconductor devices and more specifically to stacked volatile and nonvolatile memory dies for semiconductor packaging. Background Art

[0002] Microelectronic devices, such as memory devices, microprocessors, and other electronic devices, typically include one or more semiconductor dies mounted to a substrate and packaged in a protective cover. The semiconductor die includes functional features, such as memory cells, processor circuits, imager devices, interconnect circuitry, etc. In order to meet the ever-increasing demand for size reduction, wafers, individual semiconductor dies, and / or active components are typically manufactured in batches, singulated, and then stacked on a supporting substrate, such as a printed circuit board (PCB) or other suitable substrate. The stacked dies can then be coupled to a supporting substrate by bonding wires in a shingled stack of dies (e.g., dies stacked with each die offset) and / or by through-substrate vias (TSVs) between the dies and a supporting substrate (sometimes also referred to as a package substrate). Summary of the invention

[0003] In one aspect, the present disclosure relates to a system-in-package (SiP) device comprising: a base substrate; a processing unit carried by the base substrate; and a combined high-bandwidth memory (HBM) device carried by the base substrate and electrically coupled to the processing unit through one or more traces, wherein the combined HBM device comprises: an interface die; one or more volatile memory dies carried by the interface die; one or more non-volatile memory dies carried by the one or more volatile memory dies; and a shared bus electrically coupled to each of the interface die, the one or more volatile memory dies, and the one or more non-volatile memory dies.

[0004] On the other hand, the present disclosure relates to a method, comprising: writing a copy of a partition of a data set to a non-volatile memory die in a combined HBM device to store the partition of the data set in the non-volatile memory die; generating a request for a subset of the partition of the data set; writing a copy of the subset of the partition from the non-volatile memory die to multiple volatile memory dies in the combined HBM device; reading the subset of the partition from one or more of the multiple volatile memory dies into a computer processing unit or a graphics processing unit (CPU / GPU); processing the subset of the partition at the CPU / GPU; and writing the result of processing the subset of the partition to the non-volatile memory die.

[0005] In yet another aspect, the present disclosure relates to a method comprising: writing a data set to a main memory through a set of through-silicon vias (TSVs) in a shared bus, wherein the main memory includes one or more volatile memory dies; receiving a power-off or idle request; and controlling the main memory to write the data set from the main memory to a memory extension through the set of TSVs in response to the power-off or idle request, wherein the memory extension includes one or more non-volatile memory dies. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 is a schematic diagram illustrating an environment incorporating a high bandwidth memory architecture.

[0007] Figure 2 is a schematic diagram illustrating an environment incorporating a high bandwidth memory architecture in accordance with some embodiments of the present technology.

[0008] Figure 3 is a partially schematic cross-sectional view of a system-in-package with a combined high bandwidth memory device configured in accordance with some embodiments of the present technology.

[0009] Figure 4 is a partially schematic exploded view of a combined high bandwidth memory device configured in accordance with some embodiments of the present technology.

[0010] Figure 5A is a schematic top plan view of components of a combined high bandwidth memory device configured in accordance with some embodiments of the present technology.

[0011] Figure 5B is a schematic routing diagram of signals through a combined high bandwidth memory device according to some embodiments of the present technology.

[0012] Figure 6 is a flow chart of a process for operating a combined high bandwidth memory device according to some embodiments of the present technology.

[0013] Fig. 7A and 7B is a flow chart of a process for powering down and powering up a system-in-package device, respectively, using a combined high bandwidth memory device, in accordance with some embodiments of the present technology.

[0014] Figure 8 is a partially schematic exploded view of a combined high bandwidth memory device configured in accordance with further embodiments of the present technology.

[0015] Fig. 9 is a schematic routing diagram of signals through a combined high bandwidth memory device according to some embodiments of the present technology.

[0016] The drawings are not necessarily drawn to scale. In addition, it will be understood that some of the drawings have been drawn schematically and / or partially schematically. Similarly, for the purpose of discussing some embodiments of the present technology, some components and / or operations may be separated into different blocks or combined into a single block. In addition, although the present technology may have various modifications and alternative forms, specific embodiments have been shown by way of example in the drawings and described in detail below. However, it is not intended to limit the present technology to the specific embodiments described. DETAILED DESCRIPTION

[0017] High data reliability, high-speed memory access, low power consumption, and reduced chip size are features required of semiconductor memories. In recent years, three-dimensional (3D) memory devices have been introduced. Some 3D memory devices are formed by vertically stacking memory dies and interconnecting the dies using through-silicon (or through-substrate) vias (TSVs). The benefits of 3D memory devices include shorter interconnects (which reduce circuit delays and power consumption), a large number of vertical pathways between layers (which allow wide bandwidth buses between functional blocks (such as memory dies) in different layers), and a fairly small footprint. Therefore, 3D memory devices contribute to higher memory access speeds, lower power consumption, and a reduction in chip size. Example 3D memory devices include hybrid memory cubes (HMCs) and high-bandwidth memories (HBMs). For example, HBM is a type of memory that includes a vertical stack of dynamic random access memory (DRAM) dies and an interface die (which, for example, provides an interface between the DRAM die of an HBM device and a host device).

[0018] In a system-in-package (SiP) configuration, the HBM device can be integrated with a host device (e.g., a graphics processing unit (GPU) and / or a computer processing unit (CPU)) using a base substrate (e.g., a silicon interposer that provides interconnection between the GPU / CPU and the HBM device and / or provides mechanical support for components of the SiP device, an organic material substrate, an inorganic material substrate, and / or any other suitable material), and the HBM device and the host communicate through the base substrate. Because the traffic between the HBM device and the host device resides within the SiP (e.g., using signals routed through the silicon interposer), a higher bandwidth can be achieved between the HBM device and the host device than in conventional systems. In other words, the TSVs that interconnect the DRAM dies within the HBM device and the silicon interposer that integrates the HBM device and the host device enable routing of a larger number of signals (e.g., a wider data bus) than those typically found between a packaged memory device and a host device (e.g., through a printed circuit board (PCB)). The high-bandwidth interface within the SiP enables large amounts of data to be quickly moved between the host device (e.g., GPU / CPU) and the HBM device during operation. For example, the high bandwidth channel may be on the order of 1000 gigabytes per second (GB / s, sometimes also referred to as gigabits (Gb)). It will be appreciated that such high bandwidth data transfer between the GPU / CPU and the memory of the HBM device may be advantageous in various high performance computing applications such as video rendering, high resolution graphics applications, artificial intelligence and / or machine learning (AI / ML) computing systems, and other complex computing systems and / or various other computing applications.

[0019] Figure 1 is a schematic diagram illustrating an environment 100 incorporating a high bandwidth memory architecture. Figure 1 As illustrated in FIG. 1 , environment 100 includes a SiP device 110 having one or more processing devices 120 ( 120 ) integrated with a silicon interposer 112 (or any other suitable base substrate). Figure 1 , sometimes referred to herein as one or more "hosts") and one or more HBM devices 130 ( Figure 11 ). The environment 100 additionally includes a storage device 140 coupled to the SiP device 110. The processing device(s) 120 may include one or more CPUs and / or one or more GPUs (referred to as CPU / GPU 122), each of which may include registers 124 and a level 1 cache 126. The level 1 cache 126 (also referred to herein as "L1 cache") is communicatively coupled to a level 2 cache 128 (also referred to herein as "L2 cache") via a first communication path 152. In the illustrated embodiment, the L2 cache 128 is incorporated into the processing device(s) 120. However, it will be understood that the L2 cache 128 may be integrated into the SiP device 110 separately from the processing device(s) 120. Purely by way of example, the processing device(s) 120 may be carried by a base substrate (e.g., an interposer that is itself carried by a package substrate) that is adjacent to and in communication with the L2 cache 128 via one or more signal lines therein (or other suitable signal routing lines). The L2 cache 128 may be shared by one or more of the processing devices 120 (and the CPU / GPU 122 therein). During operation of the SiP device 110, the CPU / GPU 122 may use the registers 124 and the L1 cache 126 to complete processing operations, and attempt to retrieve data from the larger L2 cache 128 whenever a cache miss occurs in the L1 cache 126. Thus, the multiple levels of cache may help speed up the average time it takes the processing device(s) 120 to access data, thereby speeding up the overall processing rate.

[0020] like Figure 1 As further described in the specification, the L2 cache 128 is communicatively coupled to the (several) HBM devices 130 via a second communication channel 154. As described, the (several) processing devices 120 (and the L2 caches 128 therein) and the (several) HBM devices 130 are carried by the silicon interposer 112 and electrically coupled to the silicon interposer 112 (e.g., integrated by the silicon interposer 112). The second communication channel 154 is provided by the silicon interposer 112 (e.g., the silicon interposer includes and routes interface signals that form the second communication channel, such as through one or more redistribution layers (RDLs)). Figure 11, the L2 cache 128 is also communicatively coupled to the memory device 140 through a third communication channel 156. As illustrated, the memory device 140 is outside the SiP device 110 and utilizes signal routing components that are not contained within the silicon interposer 112 (e.g., between the packaged SiP device 110 and the packaged memory device 140). For example, the third communication channel 156 can be a peripheral bus, such as a peripheral component interconnect express (PCIe) bus, for connecting components on a motherboard or PCB. Thus, during operation of the SiP device 110, the processing device(s) 120 can read data from and / or write data to the HBM device(s) 130 and / or the memory device 140 through the L2 cache 128.

[0021] In the illustrated environment 100, the HBM device 130 includes one or more stacked volatile memory dies 132 (e.g., DRAM dies, Figure 1 1). As explained above, the (several) HBM devices 130 may be located on the silicon interposer 112, and the (several) processing devices 120 are also located on the silicon interposer 112. Therefore, the second communication channel 154 may provide a high-bandwidth (e.g., on the order of 1000 GB / s) channel through the silicon interposer 112. In addition, as explained above, each HBM device 130 may provide a high-bandwidth channel (not shown) between the volatile memory dies 132 therein. Therefore, data may be communicated at high speed between the (several) processing devices 120 and the (several) HBM devices 130 (and the volatile memory dies 132 therein), which may be advantageous for data-intensive processing operations. Although the (several) HBM devices 130 of the SiP device 110 provide relatively high-bandwidth communications, their integration on the silicon interposer 112 has certain disadvantages. For example, each HBM device 130 may provide a limited amount of storage (e.g., each on the order of 16 GB), wherein the total storage provided by all of the HBM devices 130 may be insufficient to maintain a working data set for operations to be performed by the SiP device 110. Additionally or alternatively, the HBM device(s) 130 are comprised of volatile memory (e.g., each requires power to maintain stored data, and the data is lost once the HBM device is powered off and / or suffers an unexpected power loss).

[0022] In contrast to the characteristics of HBM device 130, storage device 140 may provide large amounts of storage (e.g., on the order of terabytes and / or tens of terabytes). The larger capacity of storage device 140 is generally sufficient to maintain a working data set for complex operations to be performed by SiP device 110. In addition, storage device 140 is generally non-volatile (e.g., composed of NAND-based storage devices (e.g., NAND flash memory, such as Figure 1 ), and thus the stored data is retained even after power is lost. However, as discussed above, the storage device 140 is located outside of the SiP device 110 (e.g., not placed on the silicon interposer 112), but is coupled to the SiP device 110 through a communication channel (e.g., PCIe) arranged on a motherboard, system board, or other form of PCB. Therefore, the third communication channel 156 may have a relatively low bandwidth (e.g., on the order of 8 GB / s), which is significantly lower than the bandwidth of the second communication channel 154. Therefore, when data is moved between the storage device 140 and the SiP device 110, processing operations involving large amounts of data (e.g., graphics rendering, AI / ML processes, and the like) that are not suitable for the storage capacity of the HBM device 130 are hindered by the low bandwidth of the third communication channel 156. In addition, power-off / power-on operations that require data to move between the storage device 140 and the SiP device 110 are hindered by the relatively low bandwidth of the third communication channel 156.

[0023] Disclosed herein are HBM devices and associated systems and methods that address the shortcomings discussed above. An HBM device may include one or more volatile memory dies (e.g., DRAM dies), one or more non-volatile memory dies (e.g., NAND dies, NOR dies, PCM dies, FeRAM dies, MRAM dies, and / or any other suitable dies), and an interface die. An HBM device may also include one or more TSVs that electrically couple the interface die, one or more volatile memory dies, and one or more non-volatile memory dies to establish a communication path between them. As described herein, the TSVs may provide a wide communication path (e.g., on the order of 1024 I / Os) between the volatile memory dies, non-volatile memory dies, and interface die of an HBM device, thereby achieving high bandwidth between them. In other words, the disclosed HBM device combines both volatile memory and non-volatile memory (referred to herein as a "combined HBM device") while providing high-bandwidth communication between memories within the combined HBM device and between the combined HBM device and a coupled host device. As explained herein, embodiments of the combined HBM device may be integrated into a SiP device that combines one or more of the combined HBM devices and one or more host devices (e.g., a processing device including a CPU and / or a GPU). The combined HBM device and the host device of the SiP device may be placed on and / or integrated with a silicon interposer, which may provide a high-bandwidth communication channel between the combined HBM device and the host device.

[0024] The volatile memory die of the combined HBM device can provide the same Figure 1The non-volatile memory die 132 discussed herein has similar functionality. That is, for example, the volatile memory die may provide DRAM-based storage of a working data set accessible via a high-bandwidth interface to one or more host devices (e.g., a CPU and / or GPU of a SiP). As described herein, the non-volatile memory die of a combined HBM device may achieve various improvements to a conventional HBM device. In some embodiments, the non-volatile memory die provides additional memory capacity that can be used by a coupled host device via a high-bandwidth communication path (e.g., the non-volatile memory die operates as a memory expansion). In the embodiment, for example, a large data set may be copied from a storage component to a memory expansion (e.g., a non-volatile memory die of a combined HBM device) via a low-bandwidth communication path (e.g., PCIe) during an initialization phase. Then, during processing, portions of the large data set may be transmitted between the non-volatile memory die and the volatile memory die of the combined HBM device via the high-bandwidth communication path of the combined HBM device based on processing portions of the large data set (e.g., a working data set) at a time. In some embodiments, the nonvolatile memory die provides nonvolatile storage of some data stored in the volatile memory die of the combined HBM device (e.g., the nonvolatile memory die operates as a nonvolatile DRAM). In such embodiments, the nonvolatile memory die may not be usable by the host device (e.g., they may not increase the memory capacity available to the host device and / or may not be used to increase their memory capacity). In such embodiments, the nonvolatile memory die operating as a nonvolatile DRAM may save data from the volatile memory die and restore data to the volatile memory die in response to specific events (e.g., power off and / or power on). For example, in response to a power off or idle request, data from either of the volatile memory die and / or cache may be stored in the nonvolatile memory die in response to a power off or idle request to store the current state of the SiP device. Because the nonvolatile memory die is available through a high bandwidth communication path, the request may be satisfied much faster than communicating the data to a separate storage component (e.g., on the order of tens of milliseconds instead of seconds). Similarly, in response to receiving a power on or wake up request, data can be moved back to the volatile memory die and / or cache(s) over the high bandwidth communication path. Thus, the saved state of the SiP can be restored and the power on request can be answered within tens of milliseconds (rather than the seconds required when data must be loaded from separate storage components).

[0025] Additional details about combined HBM devices and associated systems and methods are described below. For ease of reference, semiconductor packages (and their components) are sometimes described herein with reference to front and back, top and bottom, up and down, upward and downward, and / or horizontal planes, xy planes, vertical or z directions relative to the spatial orientation of the embodiments shown in the figures. However, it should be understood that semiconductor assemblies (and their components) can be moved to and used in different spatial orientations without changing the structure and / or function of the disclosed embodiments of the present technology. In addition, signals within semiconductor packages (and their components) are sometimes described herein with reference to downstream and upstream, front and back, and / or read and write relative to the embodiments shown in the figures. However, it should be understood that signal flows can be described in various other terms without changing the structure and / or function of the disclosed embodiments of the present technology.

[0026] Furthermore, while the memory device architecture disclosed herein is primarily discussed in the context of expanding storage capacity to improve artificial intelligence and machine learning models and / or creating non-volatile memory in dynamic random access memory (DRAM) components, those skilled in the art will appreciate that the scope of the present technology is not limited thereto. For example, the systems and methods disclosed herein may also be deployed to expand available high-bandwidth memory for various other applications that process large amounts of data (e.g., video rendering, decryption systems, and the like).

[0027] Figure 2 2 is a schematic diagram illustrating an environment 200 incorporating an HBM architecture according to some embodiments of the present technology. Similar to the environment 100 discussed above, the environment 200 includes a SiP device 210 having one or more processing devices 220 ( Figure 2 ) and one or more storage devices 240 ( Figure 2 However, with Figure 1 In contrast to the SiP device 110 described in Figure 2 The embodiment of the SiP device 210 described in the embodiment includes one or more combined HBM devices 230 ( Figure 22 (one is illustrated in FIG. 1 ), as further described below. The processing device(s) 220 and the combined HBM device(s) 230 are each integrated on an interposer 212 (e.g., a silicon interposer, another organic interposer, an inorganic interposer, and / or any other suitable base substrate) that may include one or more signal routing lines. The processing device(s) 220 are driven by a CPU / GPU 222 that includes registers 224 and an L1 cache 226. The L1 cache 226 is communicatively coupled to the L2 cache 128 via a first communication channel 252. In addition, the L2 cache 228 is communicatively coupled to the combined HBM device 230 through a second communication channel 254 and is communicatively coupled to the storage device 240 through a third communication channel 256. Still further, the second communication channel 254 may have a relatively high bandwidth (e.g., on the order of 1000 GB / s), while the third communication channel 256 may have a relatively low bandwidth (e.g., on the order of 8 GB / s).

[0028] exist Figure 2 In the embodiment illustrated in , the combined HBM device(s) 230 each include a stack of one or more volatile memory dies 232 (e.g., DRAM dies) and one or more non-volatile memory dies 234 (e.g., NAND dies, NOR dies, or other suitable memory dies). That is, the one or more volatile memory dies 232 and the one or more non-volatile memory dies 234 may be stacked vertically in the combined HBM device 230. The non-volatile memory dies 232 may be stacked vertically with the non-volatile memory dies 234. Figure 11 and 2. The volatile memory die 132 of the conventional HBM device 130 illustrated in FIG. 1 operates similarly and provides similar functionality thereto. In addition, as discussed in more detail below, each of the volatile memory die 232 and the non-volatile memory die 234 may be coupled to a high bandwidth bus in the combined HBM device 230. For example, each HBM device 230 may include a plurality of TSVs interconnecting the volatile memory die 232 and the non-volatile memory die 234 within the combined HBM device 230, thereby providing a high bandwidth bus. As discussed in more detail below, the combination of volatile memory and non-volatile memory within each combined HBM device 230 may provide certain advantages. For example, volatile memory (such as DRAM) generally provides relatively faster but lower density (e.g., storage capacity within a die footprint) access (e.g., read and write) than non-volatile memory (e.g., NAND). In contrast, non-volatile memory (e.g., NAND) generally provides high storage density, but may be relatively slow to access and may incur certain overhead (e.g., wear leveling). Thus, the volatile memory die(s) 232 may provide low latency fast communication, making data quickly available to the processing device(s) 220 of the SiP device 210 as needed. The non-volatile memory die 234 may provide a relatively large memory capacity that is "closer" to the processing device 220 (e.g., accessible within the SiP device 210 via a high bandwidth bus (e.g., TSV and second communication channel 254 of the combined HBM device 230)) than the storage device 240 (e.g., accessible via a slower channel (e.g., PCIe)). Additionally, the non-volatile memory die 234 may provide non-volatile memory capacity that is closer to the processing device 220 and / or the volatile memory die 232 than the storage device 240 and / or other non-volatile memory capacity.

[0029] Thus, for example, a relatively large data set may be communicated from the storage device 240 to the non-volatile memory die(s) 234 to initiate a processing operation (e.g., to run an AI / ML algorithm). For example, the entire data set required for the AI / ML operation may be copied from the storage device 240 to the non-volatile memory die 234. Then, a subset of the data set may be quickly communicated to the volatile memory die(s) 232 via the HBM bus 258, and then communicated to the processing device(s) 220 via the high bandwidth of the second communication channel 254 (also sometimes referred to herein as a “high bandwidth communication path”). When the processing device(s) 220 completes processing the subset, the new subset may be quickly written from the non-volatile memory die(s) 234 to the volatile memory die(s) 232 without the need to retrieve data from the storage device 240 with the attendant bottleneck in the third communication channel 256 (also sometimes referred to herein as a “low bandwidth communication path”). Furthermore, processing operations can be performed iteratively (e.g., as is often the case with AI / ML algorithms) without the need to communicate large data sets multiple times through bottlenecks. Thus, the inclusion of non-volatile memory die(s) 234 can increase the processing speed of SiP device 210, thereby increasing the functionality of environment 200. Furthermore, because communicating data over a high-bandwidth channel is more efficient than communicating data over a low-bandwidth channel, the inclusion of non-volatile memory die(s) 234 can reduce the overall power consumption of environment 200 and / or reduce the heat generated by environment 200.

[0030] Additionally or alternatively, the non-volatile memory die(s) 234 may save a copy of the data being processed and / or the overall state of the SiP device 210 in a non-volatile component. Thus, for example, the state of the volatile memory die(s) 232 need not be written between the volatile memory die(s) 232 and the storage device 240 to power down and / or power up. Instead, the state may be written to the non-volatile memory die(s) 234. Thus, a power down operation (also sometimes referred to herein as a "sleep operation" and / or "idle operation") may be completed immediately (e.g., when a copy of the data is continuously stored in the non-volatile memory die(s) 234) or nearly immediately (e.g., when copied via the HBM bus 258 in the combined HBM device 230). Similarly, a power-on operation (also sometimes referred to herein as a "wake-up operation") may write the state from the non-volatile memory die(s) 234 back to the volatile memory die(s) 232 via the HBM bus 258, rather than writing from the storage device 240 back to the volatile memory die(s) 232 via the third communication channel 256. Thus, power-off and / or power-on operations may be accelerated from several seconds to much less than a second (e.g., tens of milliseconds). Additionally or alternatively, the non-volatile memory die(s) 234 may provide protection against power loss in the environment 200. For example, because the non-volatile memory die(s) 234 are coupled to the HBM bus 258, they may continuously save data (e.g., the state of the SiP device 210) as the data is written to the volatile memory die(s) 232. In other words, the non-volatile memory die(s) 234 may operate as a non-volatile shadow of the data written to the volatile memory die(s) 232. Thus, when power is lost, the non-volatile memory die(s) 234 will retain the state of the SiP device 210 after the last read / write operation between the processing device(s) 220 and the combined HBM device(s) 230, thereby reducing the chance of losing data during operation of the environment 200 (and any electronic device into which the environment 200 is integrated).

[0031] Environment 200 may be configured to perform any of a variety of suitable computing, processing, storage, sensing, imaging, and / or other functions. For example, representative examples of systems that include environment 200 (and / or its components, such as SiP device 210) include, but are not limited to, computers and / or other data processors, such as desktop computers, laptop computers, Internet appliances, handheld devices (e.g., palmtop computers, wearable computers, cellular or mobile phones, automotive electronics, personal digital assistants, music players, etc.), tablet computers, multiprocessor systems, processor-based or programmable consumer electronics, network computers, and minicomputers. Additional representative examples of systems that include environment 200 (and / or its components) include lights, cameras, vehicles, etc. With respect to these and other examples, environment 200 may be housed in a single unit or distributed across multiple interconnected units (e.g., via a communications network), distributed across various locations on a motherboard, etc. In addition, components of environment 200 (and / or any of its components) may be coupled to various other local and / or remote memory storage devices, processing devices, computer-readable storage media, etc. Reference is made below to Figures 3 to 9 Additional details are provided regarding the environment 200 , the SiP device 210 , the architecture of the combined HBM device(s) 230 , and their operation.

[0032] Figure 3 is a partially schematic cross-sectional view of a SiP device 300 having a combined HBM device 330 configured in accordance with some embodiments of the present technology. Figure 3 , the SiP device 300 includes a base substrate 310 (e.g., a silicon interposer, another suitable organic substrate, an inorganic substrate, and / or any other suitable material), and a CPU / GPU 320 and a combined HBM device 330 each integrated with an upper surface 312 of the base substrate 310. In the illustrated embodiment, the CPU / GPU 320 and associated components (e.g., registers, L1 cache, and the like) are illustrated as a single package, and the combined HBM device 330 includes a semiconductor die stack. The semiconductor die stack in the combined HBM device 330 includes an interface die 332, one or more volatile memory dies 334 ( Figure 3 ) and one or more non-volatile memory die 336 ( Figure 3 The CPU / GPU 320 is coupled to the combined HBM device 330 via a high-bandwidth bus 340, which includes one or more routing lines 344 ( Figure 3 In various embodiments, routing line 344 may include one or more metallization layers formed in one or more RDL layers of base substrate 310 and / or one or more vias interconnecting the metallization layers and / or traces. Figure 3 Not illustrated, but to be understood, the CPU / GPU 320 and the combined HBM device 330 may each be coupled to the routing line 344 via solder structures (eg, solder balls), metal-to-metal bonds, and / or any other suitable conductive bonds.

[0033] As discussed in more detail below, the high bandwidth bus 340 may also include a plurality of through substrate vias 342 ("TSVs 342," see FIG. 342 ) extending from the interposer die 332 through the volatile memory die 334 to the non-volatile memory die 336. Figure 4 ). TSVs 342 allow each of the dies to communicate data at a relatively high rate (e.g., on the order of 1000 GB / s or more) within the combined HBM device 330 (e.g., between a volatile memory die 334 (e.g., a DRAM die) and a non-volatile memory die 336 (e.g., a NAND die)). In addition, the combined HBM device 330 may include one or more signal routing lines 341 (e.g., additional TSVs extending through the interface die 332) that couple the interface die 332 and / or the TSVs 342 to the routing lines 344. In turn, the signal routing lines 341, TSVs 342, and routing lines 344 allow the dies in the combined HBM device 330 and the CPU / GPU 320 to communicate data at a high bandwidth.

[0034] Figure 4 is a partially schematic exploded view of a combined HBM device 400 configured according to some embodiments of the present technology. For example, the combined HBM device 400 can be used as the above reference Figure 3 The combined HBM device 330 discussed. In the illustrated embodiment, the combined HBM device 400 is an interface die 410, one or more volatile memory dies 420 ( Figure 4 4) and one or more non-volatile memory die 430 ( Figure 4 In addition, the combined HBM device 400 includes a shared HBM bus 440 that communicatively couples the interface die 410, the volatile memory die 420, and the non-volatile memory die 430.

[0035] The interface die 410 may be a component that is connected between the shared HBM bus 440 and an external component (eg, Figure 3The interface die 410 may include one or more active components, such as a static random access memory (SRAM) cache, a memory controller, and / or any other suitable components. The volatile memory die 420 (also sometimes collectively referred to herein as "main memory") may be a DRAM memory die that provides low-latency memory access to the combined HBM device 400. In contrast, the non-volatile memory die 430 (sometimes referred to herein as "auxiliary memory die," "memory extension," "memory extension die," and the like) may provide a non-volatile memory device (e.g., a NAND flash device) for the combined HBM device 400. In addition, the non-volatile memory die 430 may provide a significant expansion of available memory (e.g., two, three, four, five, ten, or any other suitable increase in the memory capacity of the volatile memory die 420). In a specific non-limiting example, each of the volatile memory die 420 may provide 4 GB of memory, while the non-volatile memory die 430 may provide 64 GB of memory. Figure 4 The four volatile memory dies 420 illustrated in FIG. 4 provide 16 GB of total memory, and the nonvolatile memory die 430 provides four times the total memory of the volatile memory die 420. In this example, a SiP device (e.g., Figure 3 The SiP device 300 of FIG. 4 can avoid the delay of loading memory from an external storage component (and through a low-bandwidth communication channel) into the volatile memory die 420 for each round of processing by using the 64 GB of data that can be stored in the non-volatile memory die 430 .

[0036] In the illustrated embodiment, the shared HBM bus 440 includes a plurality of TSVs 442 (TSVs) extending from the interface die 410 through the volatile memory die 420 to the nonvolatile memory die 430. Figure 4 4, but any suitable number of TSVs is possible). Each of the TSVs 442 can support independent bidirectional read / write operations to communicate data between the dies in the combined HBM device 400 (e.g., between the interface die 410 and the volatile memory die 420, between the non-volatile memory die 430 and the volatile memory die 420, and the like). Because the TSVs 442 establish a shared HBM bus 440 between each of the dies in the combined HBM device 400, the shared HBM bus 440 can reduce (or minimize) the footprint required to establish a high-bandwidth communication route through the combined HBM device 400. Therefore, the shared HBM bus 440 can reduce (or minimize) the overall footprint of the combined HBM device 400.

[0037] Figure 5A is a schematic top plan view of components of a modular HBM device 500 configured in accordance with some embodiments of the present technology. Figure 5A As described in the above reference, the combined HBM device 500 is Figure 4 The combined HBM device 400 described is generally similar. For example, the combined HBM device 500 includes an interface die 510, one or more volatile memory dies 520 ( Figure 5A ) and one or more non-volatile memory die 530 ( Figure 5A ), and a shared HBM bus 540 that communicatively couples the interface die 510, the volatile memory die 520, and the non-volatile memory die 530.

[0038] Interface die 510 includes one or more read / write components 512 ( Figure 5A In various embodiments, read / write component 512 may couple interface die 510 to external components (eg, Figure 3 ), presenting information about the combined HBM device 500 and / or the die of the combined HBM device 500 to external components (e.g., Figure 3 The CPU / GPU 320 of the volatile memory die 520) and / or includes memory control functionality to control the movement of data between the volatile memory die 520 and / or the non-volatile memory die 530. The volatile memory die 520 each includes a memory circuit 522 (e.g., a line of capacitors and / or transistors) that can store data in a volatile array. The non-volatile memory die 530 includes a memory circuit 532 (e.g., NAND flash memory and / or NOR flash memory) having a larger storage capacity than the memory circuit 522 of the volatile memory die 520.

[0039] like Figure 5A As further described in the specification, the shared HBM bus 540 may include a plurality of TSVs 542 ( Figure 5A 32 are described in FIG. 32 ). In addition, TSVs 542 may be organized into subgroups (e.g., rows, columns, and / or any other suitable subgroups) that are selectively coupled to the die in combined HBM device 500 to simplify signal routing. Figure 5A In the embodiment illustrated in , the first memory die 520a can be selectively coupled to the first subgroup 542a of TSVs 542 (e.g., the rightmost column of TSVs 542). Therefore, read / write operations on the first memory die 520a must be performed through the first subgroup 542a of TSVs 542. Similarly, Figure 5A542b, the second memory die 520c can be selectively coupled to the second subgroup 542b of the TSVs 542, the third memory die 520c can be selectively coupled to the third subgroup 542c of the TSVs 542, and the fourth memory die 520d can be selectively coupled to the fourth subgroup 542d of the TSVs 542. In the illustrated embodiment, each of the first subgroup 542a to the fourth subgroup 542d is completely separated from the other subgroups. Therefore, each of the volatile memory dies 520, although coupled to the shared HBM bus 540, is completely individually addressed. However, it will be understood that in some embodiments, the first through fourth subgroups 542a through 542d may share one or more of the TSVs 542, and / or each of the first through fourth volatile memory dies 520a through 520d may be coupled to a shared subgroup of the TSVs 542 to allow one or more read / write operations to send data to more than one of the first through fourth volatile memory dies 520a through 520d at one time (e.g., allowing the second memory die 520b to store a copy of the data of the first memory die 520a).

[0040] exist Figure 5A In the embodiment illustrated in , the interface die 510 is coupled to each of the TSVs 542. Thus, the interface die 510 can time and help route the read / write signals to any suitable destination. Similarly, the non-volatile memory die 530 is coupled to each of the TSVs 542. Thus, the non-volatile memory die 530 can use any available subgroup of the TSVs 542 (and / or all of the TSVs 542) to send and / or receive read / write signals to store a backup of the data in the volatile memory die 520, store a large data set that can then be partitioned to the volatile memory die 520, quickly store the current state of the volatile memory die 520 during power-off and / or idle operations, quickly write the previous state of the volatile memory die 520 during power-on and / or wake-up operations, and / or perform various other suitable operations. Furthermore, because the interface die 510 and the nonvolatile memory die 530 can communicate using each of the TSVs 542 (and each of the volatile memory die 420 can communicate with the interface die 410 using a subset of the TSVs 542), the bandwidth of the communication can be greater than the bandwidth for any individual of the volatile memory die 520. For example, in the illustrated embodiment, the bandwidth between the interface die 510 and the nonvolatile memory die 530 is four times the bandwidth between the interface die 510 and any individual of the volatile memory die 520.

[0041] Figure 5B According to some embodiments of the present technology, Figure 5AA schematic diagram of signal routing of the combined HBM device 500 is shown. Figure 5B 540 , TSVs 542 are schematically represented by horizontal lines, while connections to TSVs 542 (e.g., through volatile memory die 520 and nonvolatile memory die 530) are illustrated by vertical lines that intersect the horizontal lines. It will be understood that each intersection point may represent a connection to one or more of TSVs 542 (e.g., Figure 5A The eight TSVs 542 illustrated in each of the first through fourth subgroups 542a through 542d of FIG. 5 , a single TSV, two TSVs, and / or any other suitable number of connections).

[0042] exist Figure 5B In the embodiment illustrated in FIG. 5 , the volatile memory die 520 is selectively coupled to the first subgroup 542a to the fourth subgroup 542d of the TSVs 542 while being coupled to the interface die 510. For example, the first volatile link V0 (corresponding to Figure 5A The first volatile memory die 520a of the first subgroup 542a is coupled to the second volatile link V1 (corresponding to the second volatile memory die 520b) is coupled to the first subgroup 542b, and the third volatile link V2 (corresponding to the Figure 5A The third volatile memory die 520c of the first embodiment is coupled to the third subgroup 542c, and the fourth volatile link V3 (corresponding to Figure 5A The fourth volatile memory die 520d of the first subgroup 542a is coupled to the fourth subgroup 542d. In addition, the nonvolatile memory die 530 is coupled to each of the first subgroup 542a to the fourth subgroup 542d of the TSVs 542 (e.g., at the nonvolatile link NV0). Thus, for example, a signal from the interface die 510 (e.g., from an external component (e.g., Figure 3 The read request forwarded by the CPU / GPU 320) can be forwarded to the second subgroup 542b (via the interface link I0). Therefore, the signal can only be transmitted by the second volatile link V1 (e.g., Figure 5A The second memory die 520b) and / or the non-volatile memory die 530 receive.

[0043] like Figure 5B As further described in FIG. 5 , signals in the combined HBM device 500 can move along any of three bidirectional paths between the interface die 510, one of the volatile memory dies 520, and any two of the non-volatile memory dies 530. For example, a first signal path P1 extends between the interface die 510 and the volatile memory die 520. The first signal path P1 can be used during normal operation of the combined HBM device 500 to communicate between the interface die 510 (and any suitable components therein, such as Figure 3The first signal path P2 is used to perform any number of read / write operations between the interface die 510 and the volatile memory die 520. The second signal path P2 extends between the volatile memory die 520 and the non-volatile memory die 530. The second signal path P2 can be used to write a subset of the data in the non-volatile memory die 530 (e.g., a subset of data used for the AI / ML algorithm) from the non-volatile memory die 530 to the volatile memory die 520, write the results of some computer processing to the non-volatile memory die 530, write the current state of the volatile memory die 520 to the non-volatile memory die 530, write the previous state of the volatile memory die 520 from the non-volatile memory die 530 to the volatile memory die 520, and / or perform any other suitable operation. The third signal path P3 extends between the interface die 510 and the non-volatile memory die 530. The third signal path P3 can be used to write large data sets directly to the extension die 530 (e.g., for use at various stages of AI / ML algorithms, for use when rendering graphics, etc.), store information from other locations (e.g., Figure 2 The operations described above with reference to the second and third signal travel paths P2, P3 use the same high bandwidth channels (i.e., the TSVs 542 in the shared HBM bus 540) as the first signal travel path P1, so the operations can be completed at a relatively fast rate (e.g., with the same bandwidth as the TSVs 542 in the shared HBM bus 540). Figure 2 compared to the storage device 240 performing the same read / write operation).

[0044] The bidirectional restrictions of each of the TSVs 542 in the illustrated embodiment prevent any subset of the TSVs 542 from being used for multiple operations simultaneously (e.g., simultaneously along the first and third travel paths P1, P3). However, it will be understood that in some embodiments, one or more of the signal travel paths may have multiple destinations. For example, a write operation to one of the volatile memory dies 520 along the first signal travel path P1 may simultaneously write data to the non-volatile memory die 530 along the third signal travel path P3. In addition, the first subgroup 542a may be used for a first operation (e.g., writing data from the non-volatile memory die 530 to one of the volatile memory dies 520 along the second signal travel path P2) while the second subgroup 542b is used for a second operation (e.g., writing data from the interface die 510 to the non-volatile memory die 530 along the third signal travel path P3). Still further, as discussed in more detail below, the shared HBM bus 540 may include additional TSVs and / or additional subgroups of TSVs to allow subgroups to be dedicated to various signal travel paths at the expense of a larger footprint for the shared HBM bus 540 .

[0045] Figure 6 6 is a flow chart of a process 600 for operating a combined HBM device according to some embodiments of the present technology. The process 600 may be performed by a controller (e.g., a package controller) in communication with a SiP device and / or a controller on the SiP device (e.g., Figure 3 The CPU / GPU 320) is used to load, manage and / or process data in the combined HBM device.

[0046] Process 600 may begin at block 602 by partitioning a data set. The data set may be partitioned such that each partition is suitable for a combined HBM device, a total nonvolatile storage device of a combined HBM device, an individual nonvolatile memory die of a combined HBM device, and / or a SiP device. Additionally or alternatively, the data set may be partitioned according to external requirements (e.g., according to a desired batch size of data in an AI / ML process, to maximize resource utilization during a computational process, etc.). In a specific non-limiting example, a SiP device may include the same as described above with reference to Figures 4 to 5B Four combined HBM devices (each including volatile and non-volatile memory) similar to the combined HBM device discussed, each having a non-volatile memory die providing 64GB of memory. In this example, a data set having 256GB of data can be divided into four 64GB partitions, each of which can be loaded into the non-volatile memory die of the combined HBM device for processing.

[0047] In various embodiments, partitioning the data may be based on necessary partitioning (e.g., based on the space available in each combined HBM device), desired data size (e.g., based on batch size considerations for AI / ML processes to improve generalization and convergence speed), available processing space, other computational requirements (e.g., other memory devices already in use), and / or various other suitable considerations. In some embodiments, the data set is smaller than the space available in a single non-volatile memory die of a combined HBM device, allowing process 600 to skip block 602.

[0048] As described herein, process 600 loops through blocks 604 through 608 for each combined HBM device that will receive a partition of a data set. That is, at block 604, the process selects a combined HBM device from a plurality of combined HBM devices of a SiP. At block 606, process 600 includes writing a copy of the partition of the data set to a non-volatile memory die (e.g., Figure 4 The write operation at block 606 may remove a copy of the data from a storage component (e.g., Figure 2 240) to the combined HBM device for use during processing operations. Thus, the write operation at block 606 may be limited by the bandwidth of the communication path with the storage component (e.g., as described above with reference to Figure 1 As a result, the write operation may take several seconds to complete. However, as discussed in more detail below, the data is then available via a high-bandwidth communication path, allowing the data to be used any number of times without encountering a bottleneck at the storage component.

[0049] At block 608, if the combined HBM device is not the last device designated to receive data from the set, process 600 returns to block 604 to select the next combined HBM device into which to write another partition of the data set. If each combined HBM device has been addressed, the process proceeds to block 610.

[0050] It will be appreciated that although process 600 has been described at blocks 604 through 608 as Figure 6 6. The process 600 may be an iterative process that is separate from any of the other blocks in , but the process 600 is not limited in this regard. Purely by way of example, the process 600 may simultaneously execute blocks 610 through 618 for each combined HBM device on which a partition of a data set is stored, and execute blocks 604 through 608 to continue writing the partition of the data set to additional combined HBM devices. In such embodiments, the simultaneous execution of multiple blocks in the process 600 helps reduce delays associated with low-bandwidth communication paths from storage components.

[0051] At block 610, process 600 includes receiving (or generating) a request for a subset of partitions in a particular combined HBM device. The request may be received, for example, from a CPU / GPU in a SiP device and / or any other suitable controller. Additionally or alternatively, the request may be received by a controller in the combined HBM device (e.g., by Figure 4 The interface die 410 of the HBM device is generated in anticipation of an external component needing data and / or based on a previous request from an external component. As described herein, the requested data subset may be in a non-volatile memory die (e.g., a memory expansion) of the combined HBM device.

[0052] At block 612, process 600 includes writing (or causing the subset of data to be written) a copy of the subset of data from the non-volatile memory die to one or more volatile memory dies (e.g., Figure 4 The memory die 420 of the combined HBM device is sometimes referred to herein as "main memory"). The write operation can use the shared bus in the combined HBM device to write the data using a short communication path with high bandwidth. Therefore, the write operation at block 612 can be performed in a time frame on the order of tens of microseconds, making the subset available almost immediately. Once stored in the memory die, the subset of data can be used by the controller and / or processing unit for typical purposes via the high bandwidth communication path.

[0053] For example, at block 614, process 600 includes reading a subset of the data in the volatile memory die. The read operation may move a copy of the subset to a processing unit (e.g., Figure 2 At block 616, process 600 includes processing a subset of the data (e.g., Figure 3 The CPU / GPU 320 of the processor 600 may be configured to receive the data from the processor 614. At block 618, process 600 may write the results of the processing at block 616 to the volatile memory die and / or the non-volatile memory die. The read / write operations at blocks 614, 618 may communicate data using a high bandwidth communication path (e.g., via Figure 3 and 4 6. The shared bus 340, 440 of FIG. 6A and FIG. 6B is a block diagram of a block diagram of a process 600. As a result, the subset of data is available for processing within tens of microseconds and the results of the processing are saved within tens of microseconds, so that the processing at block 616 is generally the limiting factor in the speed of the process 600 in blocks 614 to 618. After writing the results of the processing at block 618, the process 600 may return to block 614 to repeat blocks 614 to 618 any suitable number of times (e.g., when the processing at block 616 is part of an AI / ML algorithm that iteratively processes data), and / or may return to block 610 to receive (or generate) a request for a second subset of data in the non-volatile memory die and write the second subset of data to the volatile memory die for processing.

[0054] In various embodiments, the processing at block 616 may be part of an AI / ML algorithm, a video rendering process, a high-resolution graphics rendering process, various complex computer simulations, and / or any other suitable computing application. In such embodiments, the CPU / GPU will typically call and / or reference each subset of data more than once. Figures 2 to 5B The combined HBM architecture discussed allows the process 600 to avoid reading data from the storage component (and through a low-bandwidth communication channel) multiple times. Instead, the data is written once to the non-volatile memory die, then written to the volatile memory die and read any suitable number of times. While the initial write operation suffers from the bottleneck constraint of the low-bandwidth communication path from the storage component, each subsequent access to the data uses the high-bandwidth path. Therefore, each subsequent use of the data may require tens of microseconds instead of one or more seconds, thereby potentially increasing the speed of processing operations by several orders of magnitude.

[0055] Fig. 7A and 7B 1 is a flow chart of processes 700, 720 for powering off and powering on a SiP device (and / or associated semiconductor devices) using a combined HBM device, respectively, according to some embodiments of the present technology. The processes 700, 720 may be executed by a controller (e.g., a package controller) in communication with the SiP device and / or a controller on the SiP device (e.g., Figure 3 CPU / GPU 320 and / or Figure 4 This is accomplished by a controller on the interface die 410).

[0056] Fig. 7A The process 700 begins at block 702 by receiving (or generating) a command to write a data set to one or more volatile memory dies in a combined HBM device (e.g., from a memory device separate from the SiP device). The command may be, for example, in response to a user's request to launch a computing application using the SiP device.

[0057] At block 704, the process 700 writes the data set to a volatile memory die (e.g., a DRAM die) in the combined HBM device so that part (or all) of the data set is available for typical processing. Because both the nonvolatile memory die and the volatile memory die are coupled to a shared HBM bus in the combined HBM device, the process 700 can simultaneously write the data set to the nonvolatile memory die in the combined HBM device at optional block 704. By writing the data to the nonvolatile memory die, the process 700 can prevent data loss during a power outage or other sudden loss of power (e.g., a damaged power connection).

[0058] Process 700 may complete blocks 702 and 704 (collectively, block 706) any number of times during operation of the SiP to support typical processing in a semiconductor device. During processing at block 706, read / write operations may use a high-bandwidth communication path to quickly communicate data sets back and forth between the volatile memory die and the processing components, thereby allowing the read / write operations to not impose significant time constraints on the processing. In addition, in some embodiments, process 700 includes writing to a non-volatile memory die at block 706 to save the results of various processing operations, save the current state of the SiP device, the combined HBM device, and / or any related semiconductor devices. Because the non-volatile memory die is coupled to a high-bandwidth communication path (e.g., Figure 4 The save may prevent a power outage or other power outage without requiring a significant investment of time and / or suspension of processing operations, since the save may not require any additional time in some embodiments because any write operation on the volatile memory die automatically creates a save in the non-volatile memory die by virtue of the interconnection of the volatile memory die to the TSVs in the shared HBM bus.

[0059] At block 708, process 700 includes receiving a power-off request (also sometimes referred to herein as an idle request). The power-off request may be received in response to input from a user and / or another component of a system using the SiP device (e.g., to conserve power when the battery of the electronic device is low and / or in response to a power outage).

[0060] At block 710, process 700 includes placing a volatile memory die (and / or any other suitable component of a semiconductor device, such as Figure 2 The state of the L1 and L2 caches 226, 228 of the combined HBM device is written to the non-volatile memory die in the combined HBM device. Because the non-volatile memory die is coupled to a high bandwidth communication path, the write operation can be performed in tens of microseconds (for example, rather than writing data to a traditional memory device (for example, Figure 2 Thus, the SiP device can comply with the power-off request within tens of microseconds, thereby allowing the semiconductor device to save power, reduce data loss when powered off, and / or otherwise shut down quickly when requested.

[0061] Relatedly, Figure 7BThe process 720 may begin at block 722 by receiving a power on request (also sometimes referred to herein as a wake up request). The power on request may be received in response to input from a user and / or another component of a system using the SiP device (e.g., another controller in a semiconductor device). And at block 724, the process 720 may read / write the previous state of the SiP device from the non-volatile memory die to the volatile memory die and / or any other suitable component (e.g., Figure 2 226, 228). Similar to the discussion above, because the non-volatile memory die is coupled to a high-bandwidth communication path, the SiP device (and corresponding semiconductor device) can respond to a power-on request within tens of microseconds (e.g., rather than the one or more seconds required to read / write from a traditional storage component). As a result, the SiP device (and corresponding semiconductor device) can be ready for computing activities significantly faster than conventional devices.

[0062] Figure 8 FIG. 8 is a partially schematic exploded view of a modular HBM device 800 configured in accordance with another embodiment of the present technology. Figure 8 As described in the above reference, the combined HBM device 800 is Figure 4 The combined HBM device 400 discussed is generally similar. For example, the combined HBM device 800 includes an interface die 810, one or more volatile memory dies 820 ( Figure 8 ) and one or more non-volatile memory die 830 ( Figure 8 ). The combined HBM device 800 includes a shared HBM bus 840 with a plurality of TSVs 842 extending communicatively between and coupled to an interface die 810, a volatile memory die 820, and a volatile memory die 830.

[0063] However, if Figure 8 As further described in the combined HBM device 800, one or more additional dies 850 ( Figure 8 800) and an auxiliary HBM bus 860, wherein a plurality of isolation TSVs 862 (two are illustrated) couple the additional die(s) 850 to the interface die 810. In various embodiments, the additional die(s) 850 may include a static random access memory (SRAM) die (e.g., to provide a cache to the combined HBM device 800, to act as a Figure 2The additional die 850 may be an L2 cache 228 that functions as an L3 (or higher) cache, and / or the like), a controller die or other processing unit, a logic die, and / or any other suitable component. In the illustrated embodiment, the additional die 850 is positioned between the interface die 810 and the volatile memory die 820. Such positioning may minimize the length of the auxiliary bus 860 and / or avoid requiring isolation TSVs 862 to extend through the volatile memory die 820 and / or the non-volatile memory die 830.

[0064] However, it will be understood that the additional die(s) 850 may be positioned in various other locations in the combined HBM device, such as between the volatile memory die 820 and the non-volatile memory die 830, above the non-volatile memory die 830, below the interface die 810, and / or in any other suitable location. Furthermore, the additional die 850 may also be coupled to the interface die 810 through the TSVs 842 in the shared HBM bus 840 in addition to (or in lieu of) being through the isolation TSVs 862 in the auxiliary bus 860. Embodiments that omit the auxiliary bus 860 may reduce the footprint required on the interface die 810 to interconnect each of the dies in the combined HBM device 800 while requiring all traffic to move along the TSVs 842 in the shared HBM bus 840.

[0065] Fig. 9 FIG. 9 is a schematic diagram of signal routing through a combined HBM device 900 according to another embodiment of the present technology. Fig. 9 As described in the above reference, the combined HBM device 900 is Figure 5A and 5B The combined HBM device 500 discussed is generally similar. For example, the combined HBM device 900 (illustrated schematically) includes an interface die 910, and a volatile memory die 920 and a non-volatile memory die 930 that are each communicatively coupled to the interface die 910. However, in the illustrated embodiment, the combined HBM device 900 includes a first bus 940 coupled between the interface die 910 and the volatile memory die 920 and a second bus 944 coupled between the interface die 910 and the non-volatile memory die 930.

[0066] The first bus 940 includes a plurality of first TSVs 942. As illustrated by the intersection between the horizontal line n and the vertical line m, the first TSVs 942 are selectively coupled to the volatile memory die 920 to establish individual communication paths. Thus, for example, each of the memory dies can be individually addressed by a signal (e.g., a read / write signal) from the interface die 910 (and any external components coupled thereto). In contrast, the second bus 944 includes a plurality of second TSVs 946, each coupled to the non-volatile memory die 930, thereby establishing a communication path with a relatively high bandwidth between the interface die 910 and the non-volatile memory die 930 (e.g., compared to the communication path of each of the volatile memory die 920). Including both the first and second buses 940, 944 into the combined HBM device 900 may require a larger footprint to establish a communication path between each of the dies. However, the inclusion of both the first and second buses 940, 944 into the combined HBM device 900 may allow, for example, a new data set to be written to the non-volatile memory die 930 while a subset of the previous data set is being processed via communication with the volatile memory die 920. Additionally or alternatively, the results of one computation using data in the volatile memory die 920 may be stored in the non-volatile memory die 930 while the data is being used for a second computation.

[0067] exist Fig. 9 , the first bus 940 is communicatively coupled only between the interface die 910 and the volatile memory die 920. However, it will be understood that the first TSV 942 may extend and be communicatively coupled to the non-volatile memory die 930. Thus, for example, a new data set may be loaded to the non-volatile memory die 930 via the second bus 944, and then distributed from the non-volatile memory die 930 to the volatile memory die 920 (in a subset or in full) via the first bus 940. In such embodiments, the communication paths between the non-volatile memory die 930 and the volatile memory die 920 are reduced because the signals do not have to travel from the non-volatile memory die 930 to the interface die 910, and then from the interface die 910 to the volatile memory die 920.

[0068] Additionally or alternatively, the volatile memory die 920 may be communicatively coupled to the second bus 944. In such embodiments, for example, data may be routed from the non-volatile memory die 930 to the volatile memory die 920 via the second bus 944, and then routed from the volatile memory die 920 to a processing component (e.g., Figure 3In a specific non-limiting example, each of the volatile memory dies 920 may be communicatively coupled to each of the second TSVs 946 in the second bus 944. Thus, signals common between each of the volatile memory dies 920 (e.g., start / stop commands, power-off commands, and the like) may be routed through one of the second TSVs 946 to reach each of the volatile memory dies 920 along a single communication path, while the other TSVs are used for other signals.

[0069] It will be understood from the foregoing that, for the purpose of illustration, the specific embodiments of the present technology have been described herein, but the well-known structures and functions are not shown or described in detail to avoid unnecessary ambiguity in the description of the embodiments of the present technology. To the extent that any material incorporated herein by reference conflicts with the present disclosure, the present disclosure shall prevail. Where the context permits, singular or plural terms may also include plural or singular terms, respectively. In addition, unless the word "or" is explicitly limited to only represent a single item that is mutually exclusive with other items in a list involving two or more items, the use of "or" in this list should be interpreted as including any single item in (a) the list, (b) all items in the list, or (c) any combination of items in the list. In addition, as used herein, the phrase "and / or" in "A and / or B" refers to only A, only B, and both A and B. In addition, the terms "include", "include", "have" and "with" are used throughout to indicate that at least (several) stated features are included so that any larger number of identical features and / or additional types of other features are not excluded. Furthermore, the terms "substantially," "approximately," and "about" are used herein to mean within at least 10% of a given value or limit. Purely by way of example, an approximate ratio means within 10 percent of a given ratio.

[0070] Several implementations of the disclosed technology are described above with reference to the figures. A computing device on which the described technology may be implemented may include one or more central processing units, memory, input devices (e.g., keyboards and pointing devices), output devices (e.g., display devices), memory devices (e.g., disk drives), and network devices (e.g., network interfaces). Memory and memory devices are computer-readable storage media that can store instructions that implement at least part of the described technology. In addition, data structures and message structures can be stored or transmitted via data transmission media (e.g., signals on a communication link). Therefore, computer-readable media may include computer-readable storage media (e.g., "non-transitory" media) and computer-readable transmission media.

[0071] It will also be understood that various modifications may be made without departing from the present disclosure or technology. For example, the dies in the combined HBM device may be arranged in any other suitable order (e.g., wherein (several) non-volatile memory dies are positioned between the interface die and the volatile memory die; wherein the volatile memory die is at the bottom of the die stack; etc.). In addition, it will be understood by a person of ordinary skill in the art that the various components of the present technology may be further divided into subcomponents, or the various components and functions of the present technology may be combined and integrated. In addition, certain aspects of the present technology described in the context of a particular embodiment may also be combined or eliminated in other embodiments. For example, although the use of non-volatile memory dies (e.g., NAND dies and / or NOR dies) to expand the memory of the combined HBM device is discussed herein, it will be understood that alternative memory expansion dies (e.g., larger capacity DRAM dies and / or any other suitable memory components) may be used. Although such embodiments may give up certain benefits (e.g., non-volatile storage), such embodiments may still provide additional benefits (e.g., reducing the business that experiences bottlenecks, thereby allowing many complex computing operations to be performed relatively quickly, etc.).

[0072] In addition, although the advantages associated with certain embodiments of the present technology are described in the context of those embodiments, other embodiments may also exhibit such advantages, and not all embodiments need necessarily exhibit such advantages to fall within the scope of the present technology. Therefore, the present disclosure and associated technology may encompass other embodiments not explicitly shown or described herein.

Claims

1. A system-in-package SiP device, comprising: base substrate; a processing unit carried by the base substrate; and a combined high bandwidth memory (HBM) device carried by the base substrate and electrically coupled to the processing unit through one or more traces, wherein the combined HBM device comprises: Interface die; one or more volatile memory dies carried by the interface die; one or more non-volatile memory dies carried by the one or more volatile memory dies; and A shared bus is electrically coupled to each of the interface die, the one or more volatile memory dies, and the one or more non-volatile memory dies.

2. The SiP device of claim 1, wherein the shared bus comprises a plurality of through-substrate vias extending from the interface die to the one or more nonvolatile memory dies.

3. The SiP device according to claim 2, wherein: The one or more volatile memory dies include at least a first volatile memory die and a second volatile memory die; and The plurality of through-substrate vias include at least a first subset of through-substrate vias and a second subset of through-substrate vias, wherein: A first subset of the through-substrate vias are electrically coupled between the interface die, the first volatile memory die, and the one or more nonvolatile memory dies, and A second subset of the through-substrate vias are electrically coupled between the interface die, the second volatile memory die, and the one or more nonvolatile memory dies.

4. The SiP device of claim 1 , wherein the combined HBM device further comprises: a controller die carried by the interface die beneath the one or more volatile memory dies; and One or more auxiliary through-substrate vias extending between the interface die and the controller die.

5. The SiP device of claim 1 , wherein the one or more nonvolatile memory dies are configured to provide, via the shared bus, a nonvolatile copy of data stored in the one or more memory dies that is accessible to the one or more volatile memory dies in response to a power on request.

6. The SiP device of claim 1, wherein the one or more memory dies collectively have a first storage capacity, and wherein the one or more non-volatile memory dies collectively have a second storage capacity that is at least four times the first storage capacity.

7. A method comprising: writing a copy of a partition of a data set to a non-volatile memory die in a combined HBM device to store the partition of the data set in the non-volatile memory die; generating a request for a subset of the partitions of the data set; writing a copy of the subset of the partitions from the non-volatile memory die to a plurality of volatile memory dies in the combined HBM device; reading the subset of the partitions from one or more of the plurality of volatile memory dies into a computer processing unit or a graphics processing unit CPU / GPU; processing the subset of the partitions at the CPU / GPU; and Results of processing the subset of the partitions are written to the non-volatile memory die. 8 . The method of claim 7 , further comprising generating the partitions of the data set based at least in part on a division of the data set among a plurality of HBM devices.

9. The method of claim 7, wherein the subset of the partitions is a first subset of the partitions, and wherein the method further comprises: generating a request for a second subset of the partitions of the data set; writing a copy of the second subset of the partitions from the non-volatile memory die to the plurality of volatile memory dies in the combined HBM device; reading the second subset of the partitions from one or more of the plurality of volatile memory dies into the CPU / GPU; processing the second subset of the partitions at the CPU / GPU; and Results of processing the second subset of the partitions are written to the non-volatile memory die.

10. The method of claim 9, further comprising combining the result of processing the first subset and the result of processing the second subset into a single result of processing the partition of the data set.

11. The method of claim 10, further comprising writing the single result to the non-volatile memory die to store the single result resulting from processing the partition on the non-volatile memory die.

12. The method of claim 7, wherein each memory device further comprises a shared bus communicatively coupled to the plurality of volatile memory dies and the non-volatile memory dies, wherein the copies of the subsets of the partitions are written from the non-volatile memory dies to the plurality of volatile memory dies in the combined HBM device using the shared bus, and wherein the subsets of the partitions are read from one or more of the plurality of volatile memory dies into the CPU / GPU using the shared bus.

13. The method of claim 12, wherein the copy of the partition of the data set is written to the non-volatile memory die using the shared bus.

14. A method comprising: Writing a data set to a main memory through a set of through-silicon vias (TSVs) in a shared bus, wherein the main memory includes one or more volatile memory dies; receiving a power-off or idle request; and The main memory is controlled in response to the power off or idle request to write the data set from the main memory to a memory extension through the set of TSVs, wherein the memory extension includes one or more non-volatile memory dies.

15. The method of claim 14, further comprising, prior to receiving the power off or idle request, writing a copy of the data set to the memory extension through the set of TSVs to store a backup of the data set in the memory extension.

16. The method according to claim 14, further comprising: receiving a power-on or wake-up request; and The memory extension is controlled in response to the power on or wake up request to write the data set from the memory extension back to the main memory through the set of TSVs.

17. The method of claim 14, wherein writing the data set from the memory extension back to the main memory takes less than 100 milliseconds.

18. The method of claim 14, further comprising: reading the data set from the main memory through the set of TSVs to use at least a portion of the data set in a computer processing operation; and Results of the computer processing operations are written to the main memory through the set of TSVs.

19. The method of claim 18, wherein the results of the computer processing operation are written simultaneously to the main memory and the memory extension to store a copy of the results of the computer processing operation.

20. The method of claim 19, further comprising, after power loss, controlling the memory extension to write the backup of the computer processing operations from the memory extension to the main memory through the set of TSVs.

Images