Dynamic Memory Operations

Abstract

Dynamic memory operations are described. According to the described techniques, a system includes a stack memory and one or more memory monitors configured to monitor a state of the stack memory. A system manager is configured to receive the monitored state of the stack memory from the one or more memory monitors and dynamically adjust the operation of the stack memory based on the monitored state. In one or more embodiments, the system includes a memory and at least one register configured to store a ranking for each of a plurality of portions of the memory, the ranking for each being determined based on an associated retention time of each portion of the memory. A memory controller is configured to dynamically refresh the portions of the memory at different times based on the ranking for each of the plurality of portions of the memory stored in the at least one register.

Description

[Technical Field]

[0001] (Related Applications) This application claims priority to U.S. patent application Ser. No. 18 / 333,135, entitled "Dynamic Memory Operations," filed Jun. 12, 2023, which claims priority under 35 U.S.C. § 119(e) to U.S. Provisional Patent Application Ser. No. 63 / 404,792, entitled "Memory Refresh Schemes," filed Sep. 8, 2022, and to U.S. Provisional Patent Application Ser. No. 63 / 404,796, entitled "Logic Die Computation with Memory Technology Awareness," filed Sep. 8, 2022, the disclosures of which are incorporated herein by reference in their entireties. [Background technology]

[0002] Memories, such as random-access memory (RAM), store data used by processors in computing devices. Advances in memory technology have led to the deployment of various types of memory, including various non-volatile and volatile memories, for numerous applications. Examples of such non-volatile memories include, for example, ferroelectric memory and magnetoresistive RAM, while examples of such volatile memories include dynamic random-access memory (DRAM) and other stacked variants of DRAM, including high-bandwidth memories. However, traditional configurations of these memories have limitations that can restrict their use in some deployments. For example, the compute die has little awareness of the static and dynamic characteristics of the memory die. This can lead to suboptimal system performance and power usage. [Brief explanation of the drawings]

[0003] [Figure 1]FIG. 1 is a block diagram of a non-limiting exemplary system having a memory and a controller operable to perform dynamic memory operations. [Figure 2] FIG. 1 illustrates a non-limiting example of a printed circuit board architecture for a high-bandwidth memory system. [Figure 3] FIG. 1 illustrates a non-limiting example of a stack memory architecture. [Figure 4] FIG. 1 illustrates a non-limiting example of a memory array that stores retention bits for dynamically refreshing the memory. [Figure 5] FIG. 1 illustrates an example of a retention bit structure for dynamically refreshing memory. [Figure 6] FIG. 1 illustrates a non-limiting example of another stack memory architecture. [Figure 7] FIG. 1 illustrates a non-limiting example of a non-stacked memory architecture with memory and a processor on a single die. [Figure 8] FIG. 1 illustrates a procedure in an exemplary embodiment of a dynamic memory operation. [Figure 9] FIG. 10 illustrates a procedure in an additional exemplary embodiment of a dynamic memory operation. [Figure 10] FIG. 10 illustrates a procedure in an additional exemplary embodiment of a dynamic memory operation. DETAILED DESCRIPTION OF THE INVENTION

[0004] (overview) The memory wall has been called one of the key limiters in pushing the boundaries of computation in modern systems. High bandwidth memory (HBM) and other stacked dynamic random access memory (DRAM) memories are increasingly being utilized to mitigate off-chip memory access latency and increase memory density. Despite these advances, conventional systems treat multi-tiered memories as stacked "2D" memory macros. Doing so poses fundamental limitations in terms of how much bandwidth and speed stacked "3D" memories can achieve.

[0005] To overcome these problems, dynamic memory operations are described, including memory refresh schemes and memory technology and logical die computation using array awareness. In one or more embodiments including a memory refresh scheme, the described techniques treat stacked memories as true three-dimensional (3D) memories by utilizing improved refresh mechanisms and components that take advantage of the increased yield and variability resulting from the multiple memory dies of stacked memories. Thus, rather than using a static refresh for the entire memory, the described techniques dynamically refresh individual portions of the memory (e.g., individual dies and / or individual rows or banks of dies) at different times. Doing so reduces the refresh overhead incurred in conventional off-chip memory systems, thereby enabling higher performance, e.g., memory throughput and / or power usage.

[0006] In conventional approaches, stacked memory (e.g., DRAM) is statically refreshed at a “static” or “fixed” rate, for example, according to the Joint Electron Device Engineering (JEDEC) specification. Thus, conventional approaches refresh all memory at a refresh rate corresponding to a “worst case” or “pessimistic case” refresh time, such as approximately 64 milliseconds, and the refresh rate can be reduced by refreshing memory bank by memory bank. However, this can limit instructions per cycle (IPC) and system performance, and the power overhead associated with such static refreshes is higher than with the described memory refresh scheme due to the “unnecessary” refreshes.

[0007] However, by stacking multiple DRAM dies (e.g., multiple tiers of stacked memory), the described techniques take advantage of the variation in retention times between the stacked dies, such as by taking advantage of the variation between a first retention time of a first tier of memory and a second retention time of a second tier of memory.

[0008] In one example of dynamic memory operation in which a memory refresh scheme is used, for example, the memory is characterized to associate respective retention times with different portions of the memory. The portions of the memory are then ranked based on the associated retention times. In a variant, this ranking is stored to provide knowledge of the memory's varying retention times. This knowledge of the memory's varying retention times is therefore accessible not only to the controller for dynamic refresh and memory allocation, but also to the logic die to take advantage of memory technology characteristics (e.g., resulting from both manufacturing variations and dynamic conditions such as different workloads that cause differential heating and / or voltage droop across the die). The portions of the memory are then dynamically refreshed at different times based on the ranking and, in one or more variants, based on optimal operating conditions associated with the logic die.

[0009] In another example of dynamic memory operation in which a memory refresh scheme is used, the system utilizes two or more retention bits within a portion of memory (e.g., each row) configured to represent a weak logic 1 and a weak logic 0. In at least one variation, the retention bits are configured to "leak" faster than the rest of the data stored in the associated portion, thus acting as an early canary indicating an early refresh of the particular portion of memory associated with the retention bit. In one or more embodiments, the retention bits are "polled" at predetermined time intervals, such as regular or irregular intervals (e.g., based on the occurrence of an event). In some cases, the retention bits are polled at different times and / or rates for each die of memory, or for each row or word within a die of memory. In some scenarios, such polling introduces overhead in power consumption and performance. This is because memory banks may be unavailable for reads / writes due to polling, and in some cases, an error correction code (ECC) check is performed on each data read, which can degrade system performance. However, doing so reduces the number of times each row of memory must be refreshed, because each row is dynamically refreshed only if the retention bit indicates a need for a refresh. This contrasts with traditional approaches in which each portion of memory is refreshed at a worst-case or pessimistic rate. Also notably, this regains performance while increasing memory availability.

[0010] Because "off-chip" memory (e.g., DRAM) is typically implemented with destructive reads, using retention bits is not suitable for conventional systems utilizing off-chip memory. This is because polling such conventionally designed retention bits (e.g., based on classic off-chip DRAM 1t-1C bit cell circuits) reads the retention bits and then writes them back. Doing so writes a fresh set of retention bits each time the retention bits are polled, thereby destroying the history of the data in the associated portion of the memory. Therefore, these conventional systems defeat the purpose of the indicated need for refresh.

[0011] To overcome these problems, in one or more embodiments, the retention bit is configured to allow polling without destructive reads. To do so, in at least one variation, the retention bit is implemented as a tunable bit cell that, while distinct from DRAM, mimics a DRAM bit cell to capture degradation in retention time and stored value, and acts as a canary for actual degradation of other bits containing usable data. In one or more embodiments, the retention bit is configured as a special multi-port cell (e.g., a 2T gain cell DRAM) in which the read port is separate from the write port, so that the stored data is corrupted when read. Notably, these special multi-port cells have retention times similar to off-chip DRAM, or slightly worse, to allow the retention bit to function as a canary. In one or more embodiments, such bit cells are implemented with separate read and write paths so that the stored charge is not corrupted when read. In contrast to conventionally configured retention bits, the retention bits discussed herein are configured to be polled so that they are accessed separately from the actual bits of the memory portion, so as not to trigger a read and write back of all bits in the respective portion of memory that is essentially part of a DRAM read operation.

[0012] The described technologies also include logic die calculations and memory technology and array awareness. In one or more embodiments involving logic die calculations and memory technology and array awareness, one or more dies of the memory are configured with monitors to provide feedback regarding the memory. For example, the monitors are configured to monitor various conditions (e.g., manufacturing variations, aging, thermal, and / or other environmental conditions) of the memory or portions of the memory (e.g., one or more cells, rows, banks, dies, etc.). These monitors provide feedback to the logic die (e.g., memory controller, ALU, CPU, GPU), such as feedback describing one or more of those conditions, which the logic die is configured to use for memory allocation, frequency throttling of the logic (or portions of the memory), voltage throttling of the logic (or portions of the memory), etc. Alternatively or additionally, in scenarios where the logic is overclocked and / or operates at increased voltages, in at least one variation, the system is configured to increase the refresh rate of the memory to compensate for reduced bitcell retention times due to overheating of the logic die, such as in connection with some applications configured for single-threaded performance or performance boost.

[0013] Accordingly, the described techniques further utilize such monitors, in part, to increase the yield of stacked memories and treat stacked memories as true 3D memories by utilizing the variations in state experienced in different portions of the memory, for example, due to the stacked memory's multiple memory dies and due to the location of portions of the memory relative to each other, other system components, and external sources. Thus, rather than using static refresh on the memory die or static overclocking parameters for the entire logic die, the described techniques dynamically refresh the memory at different times and / or overclock individual portions of the logic die (e.g., individual dies and / or individual rows or banks of a die). Doing so allows portions of the memory and logic under more favorable conditions (e.g., thermal or manufacturing conditions) to be utilized more by the system than portions under less favorable conditions, as those conditions are detected in real time and / or during calibration. Compared to conventional architectures where adaptive systems such as adaptive / dynamic voltage frequency scaling (AVFS / DVFS) circuit blocks do not monitor stacked memory within the die, this allows for better performance of the stacked memory and / or logic and also extends the lifespan of the hardware (e.g., stacked memory).

[0014] In addition to statically refreshing stack memory with a “worst-case” or “pessimistic” refresh time as described above, in many conventional approaches, the logic die (e.g., memory controller) is unaware of the memory state (e.g., thermal state), and the memory is unaware of the state of the logic die. Some conventional HBM approaches incorporate thermal sensors within the memory die to determine refresh issuance, e.g., 1× refresh vs. 2× refresh in an HBM system. However, these approaches are intra-memory based; the memory does not inform (e.g., provide feedback to) the logic die about their state, and the logic (e.g., central processing unit (CPU), graphics processing unit (GPU), accelerator, etc.) does not inform the memory die about its state. As a result, in various scenarios, the conventional logic die continues to utilize a portion of the stack memory under less favorable conditions rather than restricting memory-based operation to a portion under more favorable conditions, and the logic is over-constrained due to pessimistic assumptions about the memory and therefore executes under less favorable and / or less performant constraints.

[0015] Additionally, because a conventional logic die (e.g., an AVFS system in a CPU or a memory interface die) does not have information about the conditions under which different portions of memory are operating (e.g., because it is not connected to monitors in portions of the memory), conventional techniques overclock the entire logic or refresh the memory in the same manner. Thus, in various scenarios, a conventional logic die “overclocks” the logic in a way that causes some portions of the stacked memory (e.g., portions under less favorable conditions) to overheat or reduce refresh or performance, while throttling the frequency of the logic under other conditions that do not allow the logic die to operate optimally. Because conventional approaches do not provide feedback about memory conditions to the logic die with which the stacked memory interfaces (or feedback about logic die conditions to the memory die), such conventional approaches cannot achieve as high performance (e.g., throughput) or power savings as the described techniques. In short, conventional approaches do not involve dynamic exchange of information about operating conditions (e.g., thermal conditions and voltage droop conditions) between the memories and the logic die with which they interface. Rather, conventional approaches optimize the logic die and memory independently, leading to inefficiencies.

[0016] However, by configuring the memory dies of a stacked memory with monitors that detect the state of different portions of the memory, the described techniques take advantage of state variations between stacked dies, such as by throttling the voltage and / or frequency differently (e.g., for overclocking) to portions of the logic die based on different detected (e.g., thermal, manufacturing) and / or known (e.g., location relative to a portion of the logic) conditions, and / or by refreshing that portion of the memory at different times based on the conditions that are optimal for the logic die.

[0017] In some aspects, the technologies described herein relate to a system that includes a stack memory; one or more memory monitors configured to monitor a state of the stack memory; and a system manager configured to receive the monitored state of the stack memory from the one or more memory monitors and dynamically adjust operation of the stack memory based on the monitored state.

[0018] In some aspects, the techniques described herein relate to a system, wherein a system manager dynamically adjusts operation of logic dies coupled to a stack memory.

[0019] In some aspects, the techniques described herein relate to a system, wherein the monitored condition includes at least one of a thermal condition or a voltage droop condition of the stacked memory.

[0020] In some aspects, the technology described herein relates to a system, where one or more memory monitors monitor the state of different portions of stack memory.

[0021] In some aspects, the techniques described herein relate to a system, wherein the stacked memory includes a plurality of dies, and the one or more memory monitors include a memory monitor for each of the plurality of dies of the stacked memory.

[0022] In some aspects, the technologies described herein relate to a system, wherein a system manager is configured to regulate operation of a logic die by throttling frequency or voltage to prevent overheating of at least a portion of the logic die coupled to a stacked memory.

[0023] In some aspects, the techniques described herein relate to a system, wherein a system manager is configured to adjust operation of a logic die by providing a modification signal to modify a voltage or frequency for operating one or more portions of the logic die coupled to a stack memory.

[0024] In some aspects, the technology described herein relates to a system, wherein a system manager is configured to regulate operation of a stack memory by providing a modification signal to modify a voltage or frequency at which one or more portions of the stack memory operate.

[0025] In some aspects, the technologies described herein relate to a system, wherein a system manager is configured to coordinate operation of a stack memory by refreshing one or more portions of the stack memory.

[0026] In some aspects, the techniques described herein relate to a system that includes a memory; at least one register configured to store a ranking for each of a plurality of portions of the memory, wherein each respective ranking is determined based on an associated retention time of the respective portion of the memory; and a memory controller for dynamically refreshing different portions of the memory at different times based on the ranking for each of the plurality of portions of the memory stored in the at least one register.

[0027] In some aspects, the techniques described herein relate to a system in which at least one register is implemented in a memory controller or in an additional portion of an interface between a memory and a logic die.

[0028] In some aspects, the technology described herein relates to a system, wherein the memory includes dynamic random access memory (DRAM).

[0029] In some aspects, the techniques described herein relate to a system where different portions of memory correspond to different dies of a DRAM.

[0030] In some aspects, the technology described herein relates to a system in which DRAM dies are arranged in a stacked configuration.

[0031] In some aspects, the techniques described herein relate to a system where the different portions of the memory correspond to individual rows of the memory.

[0032] In some aspects, the techniques described herein relate to a system where different portions of memory correspond to individual banks of memory.

[0033] In some aspects, the techniques described herein relate to a system further including a logic die for scheduling different workloads on different portions of the memory based on a ranking for each of the portions of the memory.

[0034] In some aspects, the techniques described herein relate to a system that includes a memory, retention bits associated with respective portions of the memory that indicate whether each respective portion is ready to be refreshed, and a memory controller that polls the retention bits to determine whether each respective portion is ready to be refreshed and initiates a refresh of the portion of the memory based on the polling.

[0035] In some aspects, the techniques described herein relate to a system where retention bits are designed to allow reading without writing back.

[0036] In some aspects, the techniques described herein relate to a system where a retention bit has separate read and write paths.

[0037] 1 is a block diagram of a non-limiting exemplary system 100 having a memory and controller operable to perform dynamic memory operations. In this example, system 100 includes a processor 102 and a memory module 104. Further, processor 102 includes a core 106 and a controller 108. Memory module 104 includes memory 110. In one or more embodiments, processor 102 includes a system manager 112. Memory module 104 optionally includes in-memory processing components (not shown).

[0038] According to the described technology, the processor 102 and the memory module 104 are coupled to each other via a wired or wireless connection. Additionally, the cores 106 and the controller 108 are coupled to each other via one or more wired or wireless connections. Exemplary wired connections include, but are not limited to, a bus (e.g., a data bus), an interconnect, a through-silicon via, a trace, and a plane. Examples of devices in which the system 100 may be implemented include, but are not limited to, a server, a personal computer, a laptop, a desktop, a game console, a set-top box, a tablet, a smartphone, a mobile device, a virtual and / or augmented reality device, a wearable, a medical device, a system-on-a-chip, and other computing devices or systems.

[0039] Processor 102 is an electronic circuit that performs various operations on and / or using data in memory 110. Examples of processor 102 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), an accelerated processing unit (APU), and a digital signal processor (DSP). Core 106 is a processing unit that reads and executes instructions (e.g., of a program); example instructions include append, data move, and branch. While one core 106 is shown in the illustrated example, in variations, processor 102 includes two or more cores 106; for example, processor 102 is a multi-core processor.

[0040] In one or more embodiments, memory module 104 is a circuit board (e.g., a printed circuit board) on which memory 110 is mounted. In a variation, one or more integrated circuits of memory 110 are mounted on the circuit board of memory module 104. Examples of memory module 104 include, but are not limited to, a TransFlash memory module, a single in-line memory module (SIMM), and a dual in-line memory module (DIMM). In one or more embodiments, memory module 104 is a single integrated circuit device that incorporates memory 110 on a single chip or die. In one or more embodiments, memory module 104 is composed of multiple chips or dies that implement memory 110, stacked vertically (3D) together, arranged side-by-side on an interposer or substrate, or assembled via a combination of vertical stacking or side-by-side arrangements.

[0041] Memory 110 is a device or system used to store information, such as for immediate use within the device, by, for example, cores 106 of processor 102 and / or by in-memory processing components. In one or more embodiments, memory 110 corresponds to semiconductor memory in which data is stored in memory cells on one or more integrated circuits. In at least one example, memory 110 corresponds to or includes volatile memory, such as random access memory (RAM), dynamic random access memory (DRAM), synchronous dynamic random-access memory (SDRAM), and static random-access memory (SRAM). Alternatively or additionally, memory 110 corresponds to or includes non-volatile memory, examples of which include ferroelectric RAM, magnetoresistive RAM, flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), and electronically erasable programmable read-only memory (EEPROM).

[0042] In one or more embodiments, memory 110 is configured as a dual in-line memory module (DIMM). DIMMs include an array of dynamic random access memory integrated circuits, with the module mounted on a printed circuit board. Examples of DIMM types include, but are not limited to, synchronous dynamic random access memory (SDRAM), double data rate (DDR) SDRAM, double data rate 2 (DDR2) SDRAM, double data rate 3 (DDR3) SDRAM, double data rate 4 (DDR4) SDRAM, and double data rate 5 (DDR5) SDRAM. In at least one variation, memory 110 is configured as a small outline DIMM (SO-DIMM) according to any of the SDRAM standards mentioned above, e.g., DDR, DDR2, DDR3, DDR4, and DDR5. It should be understood that memory 110 can be configured in various ways without departing from the spirit or scope of the described technology.

[0043] In one or more embodiments, system manager 112 includes or is otherwise configured to interface with various systems capable of updating the operation of various components of system 100, examples of which include, but are not limited to, an adaptive voltage scaling (AVS) system, an adaptive voltage frequency scaling (AVFS), and a dynamic voltage frequency system (DVFS). Thus, in accordance with the described techniques, system manager 112 is configured to change the logic frequency or voltage (e.g., of one or more portions of processor 102) during calibration and / or dynamically. In at least one variation, system manager 112 (or a similar manager for memory module 104) is configured to change the frequency or voltage to one or more portions of memory module 104 during calibration and / or dynamically. System manager 112 is implemented using one or more of hardware or software. In one example, for example, system manager 112 is or includes a processor. By way of example, the processor executes one or more processes to control and manage system hardware resources, such as the power supplied to different parts of the system, the temperature throughout the system, the configuration of the hardware when the system starts, etc. Alternatively or additionally, system manager 112 is or includes a process executed to perform one or more such tasks. In at least one variation, system manager 112 is or includes firmware, such as for execution by a processor implementing system manager 112. Alternatively or additionally, the system manager includes or otherwise has access to one or more registers that are written to, for example, to request services.

[0044] In conventional approaches, stack memory (e.g., DRAM) is statically refreshed at a "static" or "fixed" rate, for example, according to the Joint Electron Device Engineering (JEDEC) specification. Thus, conventional approaches refresh all memory at a refresh rate corresponding to a "worst case" / "pessimistic case" refresh time, such as approximately 64 milliseconds. However, this can limit instructions per cycle (IPC), and due to "unnecessary" refreshes, the power overhead associated with such static refreshes is higher than for logic die computations with the described memory refresh scheme and memory technology awareness.

[0045] As an example, various conventional DDR5 configurations of DRAM have performance-power-area (PPA) limitations when accessing data off-chip. A typical DRAM bitcell consists of a 1T-1C structure, where the capacitor is formed by a dielectric layer sandwiched between conductor plates. The IPC of conventionally configured systems is limited by DRAM bandwidth and latency, such as under memory-heavy workloads. In contrast, system 100 can take advantage of variations in 3D memory (e.g., DRAM) to improve IPC by reducing refresh overhead and / or dynamically increasing (e.g., overclocking) the performance of portions of stacked memory that can handle the increase.

[0046] High-bandwidth memory (HBM) provides increased bandwidth and memory density, allowing multiple layers (e.g., tiers) of DRAM dies (e.g., 8-12 dies) to be stacked on top of each other with one or more optional logic / memory interface dies. Such memory stacks can be connected to a processing unit (e.g., a CPU and / or GPU) via a silicon interposer, as discussed in more detail below in conjunction with FIG. 2. Alternatively or additionally, such memory stacks can be stacked on top of a processing unit (e.g., a CPU and / or GPU), as discussed in more detail below in conjunction with FIG. 3. In one or more embodiments, stacking memory stacks on top of a processing unit can provide additional connectivity and performance advantages over connection via a silicon interposer.

[0047] The controller 108 is a digital circuit for managing the flow of data to and from the memory 110. By way of example, the controller 108 includes logic for reading from and writing to the memory 110 and interfacing with the cores 106, and in variants includes in-memory processing components. For example, the controller 108 receives instructions from the cores 106, including accessing the memory 110, and provides data to the cores 106, for example, for processing by the cores 106. In one or more embodiments, the controller 108 is communicatively disposed between the cores 106 and the memory module 104, such that the controller 108 interfaces with both the cores 106 and the memory module 104.

[0048] FIG. 2 shows a non-limiting example 200 of a printed circuit board architecture for a high-bandwidth memory system.

[0049] The illustrated example 200 includes a printed circuit board 202, shown in this case as a multi-layer printed circuit board. In one example, the printed circuit board 202 is used to implement a graphics card. It should be understood that the printed circuit board 202 may be used to implement other computing systems, such as a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), an accelerated processing unit (APU), and a digital signal processor (DSP), to name a few, without departing from the spirit or scope of the described technology.

[0050] In the illustrated example 200, the layers of the printed circuit board 202 also include a package substrate 204, a silicon interposer 206, a processor chip 208, a memory die 210 (e.g., a DRAM die), and a controller die 212 (e.g., a high-bandwidth memory (HBM) controller die). The illustrated example 200 also shows a plurality of solder balls 214 between the various layers. Here, the example 200 shows the printed circuit board 202 as a first layer and the package substrate 204 as a second layer, with a first plurality of solder balls 214 disposed between the printed circuit board 202 and the package substrate 204. In one or more embodiments, this arrangement is formed by depositing the first plurality of solder balls 214 between the printed circuit board 202 and the package substrate 204. Additionally, example 200 shows silicon interposer 206 as a third layer with a second plurality of solder balls 214 disposed between package substrate 204 and silicon interposer 206. In this example 200, processor chip 208 and controller die 212 are shown on the fourth layer, such that a third plurality of solder balls 214 is disposed between silicon interposer 206 and processor die 208, and a fourth plurality of solder balls 214 is disposed between silicon interposer 206 and controller die 212. In this example, memory die 210 forms an additional layer (e.g., a fifth layer) disposed “above” controller die 212. The illustrated example 200 also shows through-silicon vias 216 within each of memory die 210 and controller die 212, such as for connecting these various components.

[0051] It should be understood that the system for memory refresh schemes and / or logic die computation with memory technology awareness may be implemented using different architectures in one or more variations without departing from the spirit or scope of the described technology. For example, any of the components discussed above (e.g., printed circuit board 202, package substrate 204, silicon interposer 206, processor chip 208, memory die 210 (e.g., DRAM die), and controller die 212 (e.g., high-bandwidth memory (HBM) controller die)) may be stacked and arranged in different positions, side-by-side, or combinations thereof, in accordance with the described technology. Alternatively or additionally, these components may be configured differently than illustrated; for example, memory die 210 may include only a single die in one or more variations, the architecture may include one or more processor chips 208, etc. In at least one variation, one or more of the described components are not included in the architecture for implementing memory refresh schemes and / or logic die computation with memory technology awareness according to the described technology.

[0052] In this example 200, the processor chip 208 is shown to include a logic engine 218, a first controller 220, and a second controller 222. In variations, the processor chip 208 includes more, different, or fewer components without departing from the spirit or scope of the described technology. In one or more embodiments, such as a graphics card embodiment, the logic engine 218 is configured as a three-dimensional (3D) engine. Alternatively or additionally, the logic engine 218 is configured to perform different logic operations, e.g., digital signal processing, machine learning-based operations, etc. In one or more embodiments, the first controller 220 corresponds to a display controller. Alternatively or additionally, the first controller 220 is configured to control different components, e.g., any input / output components. In one or more embodiments, the second controller 222 is configured to control memory, which in this example 200 includes a controller die 212 (e.g., a high-bandwidth memory controller die) and a memory die 210 (e.g., a DRAM die). Thus, one or more of the second controller 222 and / or the controller die 212 correspond to the controller 108 in one or more embodiments. With this in mind, in one or more embodiments, the memory die 210 corresponds to the memory 110.

[0053] The illustrated example 200 also includes a plurality of data links 224. In one or more embodiments, the data links 224 are configured as 1024 data links, used in connection with a high-bandwidth memory stack, and / or have a speed of 500 megahertz (MHz). In one or more variations, such data links are configured differently. Here, the data links 224 are shown linking memory (e.g., the controller die 212 and the memory die 210) to the processor chip 208, e.g., to interface with the second controller 222. In accordance with the described techniques, the data links 224 can be used to link various components of the system.

[0054] In one or more embodiments, one or more of the solder balls 214 and / or various other components (not shown), such as one or more of the solder balls 214 disposed between the printed circuit board 202 and the package substrate 204, are operable to implement various functions of the system, such as implementing a Peripheral Component Interconnect Express (PCIe), providing electrical current, acting as a computing component (e.g., display) connector, etc., to name a few. In the context of another architecture, consider the following example.

[0055] 3 illustrates a non-limiting example 300 of a stacked memory architecture. The illustrated example 300 includes one or more processor chips 302, a controller 304, and a memory 306 having multiple stacked portions, e.g., dies (e.g., four dies in this example). While the memory 306 is shown with four dies in this example, in variations, the memory 306 includes more (e.g., five, six, seven, eight, or more) or fewer (e.g., three or two) dies without departing from the spirit or scope of the described technology. In one or more embodiments, the dies of the memory 306 are connected by trans-silicon vias, hybrid bonds, or other types of connections, etc. In this example 300, the dies of the memory 306 include a first tier 308 (e.g., T0), a second tier 310 (e.g., T1), a third tier 312 (e.g., T2), and a fourth tier 314 (e.g., T3). In one or more embodiments, memory 306 corresponds to memory 110. Additionally, in at least one variation, an architecture implementing one or more of the described techniques optionally includes a memory monitor 316 (e.g., an in-memory monitor), a controller monitor 318, and a monitor-aware system manager 320. By way of example, monitor-aware system manager 320 corresponds to system manager 112.

[0056] The memory monitor can be implemented in various ways in accordance with the described techniques. In one or more embodiments, for example, the memory monitor is implemented in software. Alternatively, or additionally, the memory monitor is implemented in hardware. When implemented in software, for example, the memory monitor is or uses operating system memory counters (e.g., server memory counters), such as an available bytes counter indicating how many bytes of memory are available for use, a pages per second counter indicating the number of pages retrieved from disk due to hard page faults or written to disk to free space in the working set due to page faults, a memory page faults per second counter indicating the rate of page faults for all processes, including system processes, and / or a process page faults per second counter indicating the rate of page faults for a given process. In at least one variation, the memory monitor is a process executed by a processor that polls respective portions of memory at intervals and / or runs using the respective portions of memory that the memory monitor is configured to monitor. Alternatively or additionally, the memory monitor tracks memory usage through one or more system calls and / or system application programming interfaces that provide insight into the status of the memory and / or portions of the memory. When implemented in hardware, the memory monitor is a device and / or logic that is integrated with (e.g., embedded in) the portion of memory that the memory monitor is configured to monitor. Examples of hardware implementations of a memory monitor include a logic or IP block integrated with the memory (or portion of the memory) and / or a controller integrated with the memory (or portion of the memory). In one or more hardware implementations, the memory monitor includes one or more registers and / or logic units (arithmetic logic units or ALUs).

[0057] Returning to consideration of the illustrated example 300, the processor chip 302, controller 304, and memory 306 dies are arranged in a stacked configuration such that the controller 304 is disposed on the processor chip 302, a first tier 308 of memory 306 is disposed on the controller 304, a second tier 310 is disposed on the first tier 308, a third tier 312 is disposed on the second tier 310, and a fourth tier 314 is disposed on the third tier 312. In variations, the components of a system for logic die computing and / or memory refresh schemes with memory technology awareness are arranged differently, such as partially stacked and / or partially side-by-side.

[0058] In one or more embodiments, the memory 306 corresponds to DRAM and / or high-bandwidth memory (HBM) cubes stacked on a compute chip, such as the processor chip 302. Examples of the processor chip 302 include, but are not limited to, a CPU, GPU, FPGA, or other accelerator. In at least one variation, the system includes a controller 304 (e.g., a memory interface die) stacked between the processor chip 302 and the memory 306, i.e., stacked above the processor chip 302. Alternatively or additionally, the controller 304 resides on the same die as the processor chip 302, e.g., in a side-by-side arrangement. This arrangement, when used with the described techniques, provides increased memory density and bandwidth with minimal impact on power and performance to alleviate memory bottlenecks on system performance. Such an arrangement can be used in conjunction with the described refresh techniques with various other types of memory, such as FeRAM and MRAM.

[0059] As mentioned above, in conventional approaches, stacked memory (e.g., DRAM) is statically refreshed at a "static" or "fixed" rate, for example, according to the Joint Electron Device Engineering (JEDEC) specification. Thus, conventional approaches refresh all memory at a refresh rate that corresponds to a "worst case" or "pessimistic case" refresh time, such as approximately 64 milliseconds.

[0060] By stacking multiple DRAM dies (e.g., multiple tiers of memory 306), the described techniques take advantage of variations in retention times between the stacked DRAM dies, such as by taking advantage of the variation between a first retention time of a first tier 308 of memory 306 and a second retention time of a second tier 310 of memory 306. In one or more embodiments, these variations are exploited to "buy back" performance and power.

[0061] In at least one embodiment, this is achieved by employing dynamic refresh (of stack memory 306) rather than static refresh. In one example of dynamic refresh, memory 306 is characterized by associating different retention times with portions of memory. The portions of memory are ranked based on the associated retention times. This ranking can then be stored to provide knowledge of the varying retention times of memory 306 to controller 304 for dynamic refresh and memory allocation, as well as to the logic die to take advantage of memory technology characteristics. The ranking can be stored in a register of the system. As described herein, a register corresponds to a storage unit accessible by controller 304. For example, in some cases, the ranking is stored in a register of controller 304. In other embodiments, the ranking is stored in a register or storage unit accessible by controller 304, such as a memory interface or a register implemented on the logic die. For example, the register is implemented as part of the interface between the logic die and the memory die. The portions of memory are then dynamically refreshed at different times based on the ranking.

[0062] In another example of dynamic refresh, the system utilizes two or more retention bits within a portion (e.g., each row) of memory 306 configured to represent a weak logic 1 and a weak logic 0, which is discussed in more detail below in Figures 4 and 5.

[0063] As mentioned above, in one or more embodiments, memory 306 optionally includes memory monitors 316. Here, each tier of memory 306 is shown having at least one memory monitor 316, with the first tier 308 of memory 306 being shown including two memory monitors 316. However, in variations, a portion (e.g., a die) of memory 306 includes a different number of memory monitors 316 (e.g., less than one per die, one per die, more than one per die, etc.) without departing from the spirit or scope of the described techniques. Also, in variations, memory monitors 316 are located at different physical locations of memory 306 in accordance with the described techniques.

[0064] In one or more variations in which the architecture includes a memory monitor 316, the controller 304 optionally includes a controller monitor 318, and the processor chip 302 optionally includes a monitor-aware system manager 320. In variations, the controller 304 includes more than one controller monitor 318, and the physical location of one or more controller monitors 318 within the controller 304 varies without departing from the described techniques.

[0065] Generally, memory monitor 316 is configured to detect one or more conditions of a portion of memory, such as a thermal condition (e.g., thermal sensor), a voltage droop condition, and / or memory retention. In one or more embodiments, memory monitor 316 detects one or more conditions of a portion of memory that provides input to a logic die, e.g., processor chip 302, or one or more portions thereof. Similarly, controller monitor 318 is configured to detect one or more conditions of controller 304 or portions of controller 304. The detected conditions are communicated to monitor-aware system manager 320.

[0066] The monitor-aware system manager 320 statically and / or dynamically adjusts the operation of the memory 306 (or portions of the memory) based on the detected conditions. Alternatively or additionally, the monitor-aware system manager 320 dynamically adjusts the operation of the processor chip 302 (e.g., logic die). For example, the monitor-aware system manager 320 provides a modification signal to change one or more of the voltage or frequency at which one or more portions of the memory 306 operate, e.g., to overclock one or more portions and not overclock one or more other portions, and / or to overclock one or more portions of the memory differently from one or more other portions of the memory. Alternatively or additionally, the monitor-aware system manager 320 provides a modification signal to change one or more of the voltage or frequency at which one or more portions of the processor chip 302 operate, e.g., to overclock one or more portions and not overclock one or more other portions, and / or to overclock one or more portions of the processor chip 302 differently from one or more other portions of the memory. Alternatively or in addition, the monitor recognition system manager 320 updates the refresh of one or more portions of the memory 306, for example, updating the refresh of one or more portions of the memory 306 differently from one or more other portions of the memory 306.

[0067] In memories (e.g., DRAMs), retention time can vary for a variety of reasons, such as static manufacturing defects and variations, dynamic thermal variations, dynamic row hammering (e.g., of adjacent rows causing additional leakage from storage capacitors within a bitcell (DRAM 1T-1C bitcell)), and aging of components within the memory that degrades memory retention over the life of the system.

[0068] In the context of a stacked arrangement such as the arrangement shown in example 300, for example, the first tier 308 of memory 306 is likely to experience the largest thermal gradient because the first tier 308 is closest to the compute die (e.g., processor chip 302) and because, in many cases, the logic die (of the compute chip) has higher activity and often generates more heat than the memory die. Additionally, in one or more embodiments, memory cells within a particular portion of memory 306, such as a portion of memory 306 that is physically closer to a hot logical IP block of processor chip 302, also generate more heat than other functional blocks, which causes a larger gradient in retention time variation due to thermal effects. An example of such a portion of memory 306 includes one or more cells (e.g., DRAM cells) of the first tier 308 of memory 306 that are located above a logical IP block of processor chip 302, such as an execution unit and a scheduler. Unlike conventional approaches that do not track thermal or voltage droop conditions across individual memory dies, in one or more variations, the system tracks thermal and / or voltage droop conditions across individual memories by configuring memory dies to include multiple memory monitors 316 (e.g., thermal, retention, etc.) and / or by detecting thermal and / or voltage droop conditions on one memory die in part by using a memory monitor 316 on another memory die.

[0069] According to the described techniques, the monitor-aware system manager 320 is configured to dynamically or adaptively scale voltage and / or frequency. The monitor-aware system manager 320 receives information from one or more of the memory monitors 316 and / or one or more of the controller monitors 318, which capture temperature gradients within one or more of the dies. Based on the information, the monitor-aware system manager 320 throttles frequency and / or voltage (e.g., for a portion of the processor chip 302), such as to prevent the die from overheating. The monitor-aware system manager 320 also derives voltage-frequency relationships to optimize system performance (e.g., maximize performance) and power (e.g., minimize power usage).

[0070] In one or more embodiments, the described techniques use a monitor-aware system manager 320 to account for multi-tier thermal effects (of memory 306), such as by (1) integrating a controller monitor 318 (e.g., thermal sensors) into the controller 304 (e.g., memory interface logic) and (2) integrating a memory monitor 316 (e.g., thermal sensors and / or retention monitors) into one or more tiers of memory 306 in single or multiple locations on the memory die to capture changing heat sources below (and / or above) the memory tiers, since different logical IP blocks may correspond to different heat sources and have different thermal effects due to those heat sources.

[0071] In the context of example 300, for example, the first tier 308 of memory 306 is more likely to be affected by workload, logic die heating, and / or degradation over time in different portions of memory 306 than other tiers. In one or more embodiments, one or more of the memory monitors 316 are calibrated at wafer sort or after packaging and fused to account for the unique "worst case" retention time of each tier. In a variation, one or more of the memory monitors 316 are fused within tiers of memory 306 at different granularities to capture changing thermal variations across tiers (e.g., across the first tier 308 of memory 306).

[0072] Based on this, the monitor-aware system manager 320 determines whether information from the memory monitor 316 indicates that thermal and / or manufacturing variations have caused the retention of the memory 306 (or a portion of the memory 306) to drop below a threshold retention time. In one or more embodiments, the monitor-aware system manager 320 dynamically tracks information from the memory monitor 316 during system operation to capture the effects of memory aging that affect bitcell retention times. This includes adjusting the memory monitor 316 during operation with updated retention times for tiers of memory 306 based on aging and / or row hammering. In such an embodiment, the system sets thresholds to control the monitor-aware system manager 320 (e.g., an AVFS system).

[0073] In conventional approaches, retention time is typically considered to be a single value for the entire memory (e.g., the entire DRAM or HBM), such as in accordance with JEDEC. In contrast to conventional approaches, the described techniques integrate one or more retention monitors (e.g., memory monitor 316) for logic 0 and logic 1, e.g., into each tier of memory 306. In one or more embodiments, the described techniques determine a retention preference (e.g., per die) for each of multiple memory tiers, such as during calibration. This retention preference (for logic 0 or logic 1) per die is stored, e.g., in a register file of controller 304. In at least one variation, logic is integrated into controller 304 (e.g., memory interface controller) to assign data that is primarily 1 to a tier with a preference for logic 1 and assign data that is primarily 0 to a tier with a preference for logic 0. The system determines whether the data is primarily 0 or 1 using one or more techniques, such as by calculating a checksum on the data or a derivative of the checksum. Additionally or alternatively, the memory interface / controller die (e.g., controller 304) includes logic to convert the data to include more zeros or ones. For example, the memory interface / controller die converts the data to include more zeros than ones if retention of a logical zero is longer than retention of a logical one in memory 306 (or a tier). As an example, the logic converts the data by storing the one's complement of the data if dominated by an undesired value (e.g., not a value the tier has a preference for) and by adding a bit to indicate whether the true value or the complement value is stored.

[0074] In one or more embodiments, the described techniques assign various data and / or applications to operate using different portions of memory 306, such as different tiers of memory 306. This is because some types of data and applications result in more frequent data updates and / or lower data residency than others. As an example, a GPU frame buffer is typically updated more frequently than data written by the CPU. In at least one variation, the system (e.g., controller 304) includes logic that causes the controller 304 to assign data maintained by the GPU frame buffer to a tier of memory 306 determined to have lower retention. In operation, such data is refreshed more frequently by the corresponding application and is therefore paired with a portion of memory having characteristics of the data, e.g., a condition (e.g., lower retention), that are complementary to the data's more frequent refresh. This is in contrast to data that is refreshed less frequently, such as data that is refreshed as overhead, such as CPU data, which is generally updated less frequently. In one or more embodiments, the controller 304 allocates such CPU data to portions (e.g., tiers) of the memory 306 determined to have relatively high retention. In at least one variation, various workloads involving accesses to memory are profiled based on characteristics of memory accesses associated with executing those workloads (e.g., frequency of updates). The workloads are then allocated to portions of memory based on the profiling, such as allocating workloads to portions of memory that operate under conditions that are complementary to the workload's characteristics (or more complementary than the characteristics of other workloads that are better suited to be allocated to different portions of memory). In one or more embodiments, for example, the logic die schedules different workloads in different portions of memory (e.g., through a scheduler implementation) based on ranking, where the ranking indicates retention time. During operation, some workloads are frequently modified and therefore can tolerate bitcells with lower retention times.As an example, workloads implemented on a GPU involve accessing memory much more frequently and therefore may tolerate bit cells of memory (e.g., portions of memory) having shorter retention times than other types of workloads and / or operations.

[0075] In various scenarios, applications such as probabilistic computing tolerate least significant bit (LSB) errors. In one or more embodiments, the monitor recognition system manager 320 ranks portions of the memory 306. For example, the monitor recognition system manager 320 ranks bits, rows, banks, and / or dies of the memory 306. The monitor recognition system manager 320 also stores data based on least or most significant bit characteristics and based on the monitored state of the portion of the memory 306. As an example, the monitor recognition system manager 320 stores LSB data in a portion of the memory with lower retention (according to the ranking) and most significant bit (MSB) data in a portion of the memory with higher retention (according to the ranking). In one or more embodiments, the dies of the memory 306 are stacked on top of each other and on the processor chip 302 such that functional blocks that generate more heat are aligned with the LSB bits of the memory 306 stacked on top.

[0076] While stacked configurations having multiple memory dies are discussed above, it should be understood that in one or more embodiments, the memory may include only a single die stacked on the controller 304 and / or processor chip 302. In this context, consider FIG.

[0077] FIG. 4 shows a non-limiting example 400 of a memory array that stores retention bits for dynamically refreshing the memory.

[0078] The memory array of example 400 utilizes "retention bits" associated with different portions of the memory. For example, example 400 shows a first portion 402 and a second portion 404 of memory 306. The first and second portions may correspond to different portions of memory 306, such as different rows of memory, different dies of memory 306, different banks of memory 306, etc. In this example, first portion 402 includes a first retention bit 406 and a second retention bit 408, and second portion 404 also includes a first retention bit 410 and a second retention bit 412.

[0079] In one or more embodiments, the first and second retention bits of each portion represent a weak logic one and a weak logic zero, respectively. The retention bits are configured to "leak" faster than the rest of the data stored in the associated portion; thus, the retention bits act as early canaries, indicating that the particular portion of memory associated with the retention bit is ready to be refreshed early. In one or more embodiments, the retention bits are "polled" at predetermined time intervals, for example, by the controller 304. Notably, such polling may result in overhead in power consumption. However, doing so reduces the number of times each row of memory must be refreshed, since each row is dynamically refreshed only when the retention bit indicates that the row is ready to be refreshed. Notably, this regains performance while increasing memory availability.

[0080] Typically, "off-chip" memory (e.g., DRAM) is implemented with destructive reads, and therefore using retain bits may not be suitable for conventional systems that utilize off-chip memory. This is because polling the retain bits causes them to be read and then written back. Doing so writes a fresh set of retain bits each time they are polled, thereby destroying the history of the data in the associated portion of memory.

[0081] To overcome this problem, retention bits are configured to allow polling without destructive reads. To do so, retention bits are implemented as bit cells distinct from DRAM, but can mimic DRAM bit cells to capture retention time and degradation of stored values. In one or more embodiments, retention bits are configured as special two-port cells (e.g., 2T gain cell DRAM) so that stored data is not destroyed when read. Notably, these special two-port cells have retention times similar to off-chip DRAM, or slightly worse, to allow retention bits to function as canaries. In contrast to conventionally configured DRAM, these bit cells are implemented with separate read and write paths so that stored charges are not destroyed when read. Retention bits are configured to be accessed and polled separately from the actual bits of a memory portion, so as not to trigger a read and write back of all bits in the respective portions of memory as part of a DRAM read operation.

[0082] FIG. 5 shows an example retention bit structure 500 for dynamically refreshing a memory. The retention bit structure of FIG. 5 is based on a 2T gain cell DRAM circuit. Other bits in the DRAM WL are coupled to the WWL. Therefore, polling during refresh only allows reading via the RWL without disturbing the SN on the retention bits or triggering writebacks on the DRAM row wordline bits.

[0083] One or more embodiments employ a hybrid refresh technique that uses retention bits to characterize a discrete retention time for each memory tier and refresh the DRAM stack at a discrete refresh rate assigned to each tier. That is, static refresh may occur within a die, but each stack die has its own designated / required refresh time. For example, tier 0 is refreshed at refresh rate "r0," tier 1 is refreshed at an "r1" rate (different from r0), and so on. This can be characterized in a "calibration" phase and dynamically at checkpoints inserted to recalibrate the tier retention times, capturing how these retention times change over time for the memory tiers.

[0084] In one or more embodiments, the use of retention bits is extended to protect against row hammering. For example, the retention bits described above store time-dependent history and "leakage" in the bitcell, thus providing an early indication of leakage also caused by row hammering in adjacent rows. Instead of tracking row hammering within the memory controller, which is the case when static refresh is employed, polling of retention bits during dynamic refresh at the intra-tier level catches row hammering-related "leakage" and issues refreshes through polling.

[0085] Also, soft error ECC checking can be used as a proxy to determine when to initiate a refresh. For example, an in-memory processing component can perform ECC at the bank level and use this to perform refresh at the bank level, or vice versa (refresh polling can be used to feed the ECC check).

[0086] FIG. 6 shows a non-limiting example 600 of another stacked memory architecture. The illustrated example 600 includes one or more processor chips 602, a controller 604, and a memory 606. In at least one variation, the memory 606 is a non-volatile memory, such as a ferroelectric RAM or a magnetoresistive RAM. Alternatively, the memory 606 is a volatile memory, examples of which are described above. In a variation, the components (e.g., the one or more processor chips 602, the controller 604, and the memory 606) are connected in any of a variety of ways, such as those discussed above. With this in mind, in one or more scenarios, the memory 606 corresponds to memory 110. In at least one variation, the stacked memory architecture optionally includes a memory monitor 608 (e.g., an in-memory monitor, such as a thermal sensor and / or a retention monitor), a controller monitor 610, and a monitor-aware system manager 612. In one or more embodiments, the monitor-aware system manager 612 corresponds to the system manager 112.

[0087] In this example 600, the processor chip 602, controller 604, and memory 606 are arranged in a stacked configuration, such that the controller 604 is disposed on the processor chip 602 and the memory 606 is disposed on the controller 604. As noted above, the components of the system for logic die computing and / or memory refresh scheme with memory technology awareness may be arranged differently in variations without departing from the spirit of the described technology. In variations in which the architecture includes a memory monitor 608, a controller monitor 610, and a monitor-aware system manager 612, these components include the same and / or similar functionality as discussed above with respect to FIG. 3.

[0088] In one or more embodiments in which memory 606 is non-volatile memory, memory 606 has a higher temperature tolerance than one or more volatile memory embodiments. Generally, monitor recognition system manager 612 is configured to throttle frequency and voltage based on cooling / packaging and heat generated by the logic die, as indicated by information from memory monitor 608. Because various non-volatile memories have higher temperature tolerances, in at least one variation, monitor recognition system manager 612 is configured to overdrive voltage and frequency further than one or more volatile memory configurations. In such a scenario, monitor recognition system manager 612 receives input of the thermal characteristics of the non-volatile memory, which are dynamically calibrated at wafer sort / packaging steps and / or at periodic checkpoints, thereby achieving higher system performance and lower power usage than other approaches utilizing stacked memory configurations with non-volatile memory.

[0089] As mentioned above, in one or more embodiments, the described techniques determine a preferred retention of a memory die (e.g., memory 606), such as during calibration. This preferred retention of the die (to logic 0 or logic 1) is stored, for example, in a register file of controller 604. In at least one variation, monitor-aware system manager 612 tunes voltage levels to boost writes of non-preferred values ​​to improve overall performance with minimal power overhead. As another exemplary arrangement of components, consider the following example of FIG. 7.

[0090] FIG. 7 shows a non-limiting example 700 of a non-stacked memory architecture having memory and processors on a single die. The illustrated example 700 includes one or more processor chips 702, a controller 704, and a memory 706. In at least one variation, the memory 706 is a non-volatile memory, such as a logic-compatible ferroelectric RAM or magnetoresistive RAM. Alternatively, the memory 706 is a volatile memory, examples of which are described above. In variations, the components (e.g., one or more processor chips 702, the controller 704, and the memory 706) are connected in any of a variety of ways, such as those discussed above. In one or more embodiments, the architecture includes a memory monitor 708 (e.g., an in-memory monitor, such as a thermal sensor and / or a retention monitor) and a monitor-aware system manager 710. Although not shown, in one or more embodiments, the controller 704 also includes a controller monitor.

[0091] In at least one example, such as the illustrated example 700, one or more processor chips 702, controller 704, and memory 706 are arranged side-by-side on a single die, e.g., each of these components is arranged on the same die. For example, controller 704 is connected in a side-by-side arrangement with processor chip 702, memory 706 is connected in a side-by-side arrangement with controller 704, and controller 704 is arranged between memory 706 and processor chip 702. In variations, the components of a system for a memory refresh scheme and / or logic die computing with memory technology awareness are arranged in a different side-by-side arrangement (or partial side-by-side arrangement) without departing from the spirit or scope of the described technology. In one or more embodiments, memory monitor 708 and monitor-aware system manager 710 include functionality similar to the functionality discussed above with respect to FIG. 3.

[0092] FIG. 8 illustrates a procedure in an embodiment of an example 800 of dynamic memory operations.

[0093] The state of the stack memory is monitored by one or more memory monitors (block 802). The monitored state of the stack memory is communicated by the one or more memory monitors to a system manager (block 804). The operation of the stack memory is adjusted by the memory monitors based on the monitored state (block 806).

[0094] FIG. 9 illustrates a procedure in an embodiment of an additional example 900 of dynamic memory operations.

[0095] The memory is characterized to associate different retention times with portions of the memory (block 902). By way of example, portions of the memory 306 (e.g., rows, dies, or banks) may be characterized to associate respective retention times with different portions of the memory 306.

[0096] The portions of the memory are ranked based on associated retention times (block 904). By way of example, portions of the memory 306 (e.g., rows, dies, or banks) are ranked based on associated retention times determined from the memory characterization performed in block 802.

[0097] The portions of memory are dynamically refreshed at different times based on the ranking (block 906). By way of example, different portions of memory 306 are refreshed and banked based on the ranking. Doing so contrasts with the static refresh performed by conventional systems, in which all portions of memory are refreshed simultaneously.

[0098] FIG. 10 illustrates a procedure in an embodiment of an additional example 1000 of dynamic memory operations.

[0099] Retention bits associated with individual portions of memory are polled to determine whether to refresh the individual portions of memory (block 1002). According to the techniques discussed herein, the polling causes the retention bits to be read without writing back, which stores time-dependent history and leakage in the bit cells. The individual portions of memory are refreshed at different times based on the polling (block 1004).

[0100] It should be understood that many variations are possible based on the disclosure herein, and although features and elements are described above in particular combinations, each feature or element can be used alone without the other features and elements, or in various combinations with or without the other features and elements.

[0101] The various functional units illustrated in the figures and / or described herein (including, where appropriate, memory 110, controller 108, and core 106) may be implemented in any of a variety of different ways, such as hardware circuits, software or firmware running on a programmable processor, or any combination of two or more of hardware, software, and firmware. The provided methods may be implemented in any of a variety of devices, such as a general-purpose computer, a processor, or a processor core. Suitable processors include, by way of example only, a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), a graphics processing unit (GPU), a parallel-accelerated processor, multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), and / or a state machine.

[0102] In one or more embodiments, the methods and procedures provided herein may be implemented in a computer program, software, or firmware embodied in a non-transitory computer-readable storage medium for execution by a general purpose computer or processor. Examples of non-transitory computer-readable storage media include read-only memory (ROM), random-access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs).

Claims

1. 1. A system comprising: Stack memory and one or more memory monitors configured to monitor the state of the stack memory; a system manager configured to receive monitored states of the stack memory from the one or more memory monitors and dynamically adjust operation of the stack memory based on the monitored states. system.

2. the system manager dynamically adjusts operation of logic dies coupled to the stack memory; The system of claim 1.

3. the monitored condition includes at least one of a thermal condition or a voltage droop condition of the stacked memory; The system of claim 1.

4. the one or more memory monitors monitor the state of different portions of the stack memory; The system of claim 1.

5. the stacked memory includes a plurality of dies, and the one or more memory monitors include a memory monitor for each of the plurality of dies of the stacked memory. The system of claim 1.

6. the system manager is configured to regulate operation of the logic die coupled to the stacked memory by throttling frequency or voltage to prevent overheating of at least a portion of the logic die. The system of claim 1.

7. the system manager is configured to adjust operation of the logic die by providing a modification signal to modify a voltage or frequency at which one or more portions of the logic die coupled to the stack memory operate. The system of claim 1.

8. the system manager is configured to regulate operation of the stack memory by providing a modification signal to modify a voltage or frequency at which one or more portions of the stack memory operate. The system of claim 1.

9. the system manager is configured to coordinate operation of the stack memory by refreshing one or more portions of the stack memory. The system of claim 1.

10. 1. A system comprising: Memory and at least one register configured to store a ranking for each of a plurality of portions of the memory, the ranking for each being determined based on an associated retention time of each portion of the memory; a memory controller that dynamically refreshes different portions of the memory at different times based on the ranking for each of the plurality of portions of the memory stored in the at least one register. system.

11. the at least one register is implemented in the memory controller or in an additional portion of an interface between the memory and a logic die; The system of claim 10.

12. at least one of the memories comprises dynamic random access memory (DRAM), and different portions of the memory correspond to different dies of the DRAM, or the DRAM dies are arranged in a stacked configuration; The system of claim 10.

13. the different portions of the memory corresponding to individual rows of the memory. The system of claim 10.

14. the different portions of the memory correspond to individual banks of the memory. The system of claim 10.

15. a logic die for scheduling different workloads on different portions of the memory based on the ranking for each of the plurality of portions of the memory; The system of claim 10.

Images