Memory device with on-die cache

Abstract

The present application relates to a memory device with on-die cache memory. An example memory sub-system includes a plurality of bank groups, where each bank group comprises a plurality of memory banks, a plurality of row buffers, where two or more of the plurality of row buffers are associated with each memory bank, a cache memory comprising a plurality of cache lines, processing logic communicatively coupled to the plurality of bank groups and the plurality of row buffers, the processing logic to perform operations comprising receiving an activate command specifying a row of a memory bank of the plurality of memory banks, fetching data from the specified row to a row buffer of the plurality of row buffers, and copying the data to a cache line of the plurality of cache lines.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure generally relate to memory systems, and more particularly to implementing a memory device (e.g., a dynamic random access memory (DRAM) device) with on-die cache memory. BACKGROUND

[0002] A DRAM device includes a plurality of memory banks grouped into bank groups, such as sixteen memory banks grouped into four bank groups. Each memory bank is a memory array including a plurality of memory cells, such that each memory cell is capable of storing information of one or more bits depending on the memory cell type. SUMMARY

[0003] In one aspect, the present application provides a memory device comprising: a plurality of bank groups, wherein each bank group comprises a plurality of memory banks; a plurality of row buffers, wherein at least one row buffer of the plurality of row buffers is associated with each memory bank of the plurality of memory banks; a cache memory comprising a plurality of cache lines; processing logic communicatively coupled to the plurality of bank groups and the plurality of row buffers, the processing logic to perform operations comprising: receiving an activate command specifying a row of a memory bank of the plurality of memory banks; fetching data from the specified row to a row buffer of the plurality of row buffers; and copying the data to a cache line of the plurality of cache lines.

[0004] In another aspect, the present application further provides a system on a chip comprising: a plurality of bank groups, wherein each bank group comprises a plurality of memory banks; a plurality of row buffers communicatively coupled to the plurality of bank groups; a cache memory comprising a plurality of cache lines; content addressable memory (CAM) circuitry to translate a memory address to one of: an identifier of a particular row of a particular memory bank of the plurality of memory banks or an identifier of a particular line of the cache memory; processing logic communicatively coupled to the plurality of bank groups and the plurality of row buffers, the processing logic to perform operations comprising: receiving a read command specifying a memory address; translating the memory address to an identifier of a cache line of the plurality of cache lines by the CAM circuitry; and reading data from the cache line.

[0005] In yet another aspect, the application further provides a method comprising: receiving, by a memory device, an activate command specifying a cache hint and a row of a memory bank of a plurality of memory banks of the memory device; fetching data from the specified row to a row buffer of a plurality of row buffers of the memory device; copying the data to a cache line of a plurality of cache lines of the memory device based on the cache hint interpreted in view of a cache policy associated with the memory device. BRIEF DESCRIPTION OF DRAWINGS

[0006] The present disclosure will be more fully understood from the following detailed description, taken in connection with the accompanying drawings, in which:

[0007] Figures 1A-1B A high-level component diagram of an example memory subsystem implemented in accordance with aspects of the present disclosure is schematically illustrated.

[0008] Figure 2 Interactions of a host and a memory subsystem implementing an activate (ACT) command augmented by a cache hint are schematically illustrated in accordance with aspects of the present disclosure.

[0009] Figure 3 Interactions of a host and a memory subsystem implementing a precharge (PRE) command augmented by a cache hint are schematically illustrated in accordance with aspects of the present disclosure.

[0010] Figure 4 Interactions of a host and a memory subsystem implementing a read with automatic precharge (RDA) command augmented by a cache hint are schematically illustrated in accordance with aspects of the present disclosure.

[0011] Figure 5 Interactions of a host and a memory subsystem implementing a write with automatic precharge (WRA) command augmented by a cache hint are schematically illustrated in accordance with aspects of the present disclosure.

[0012] Figure 6 A flow diagram of an example method of a memory subsystem implementing memory access operations operated in accordance with embodiments of the present disclosure.

[0013] Figure 7 An example computing system including a memory subsystem implemented in accordance with some embodiments of the present disclosure is illustrated.

[0014] Figure 8 A block diagram of an example host computer system in which embodiments of the present disclosure can operate is illustrated. DETAILED DESCRIPTION

[0015] Embodiments of the present disclosure relate to implementing a memory device, such as a dynamic random access memory (DRAM) device, with on-die cache memory.

[0016] DRAM devices include a plurality of memory banks grouped into bank groups, such as sixteen memory banks grouped into four bank groups. Each memory bank is a memory array including a plurality of memory cells, such that each memory cell is capable of storing information of one or more bits depending on the memory cell type. Memory cells include a capacitor that holds a charge and a transistor that acts as a switch to control access to the capacitor. Thus, a memory cell can be programmed (written) by applying a particular voltage, which results in a charge held by the capacitor. Memory cells are connected by a word line, which is a wire electrically connected to a control gate of the memory cell, and a bit line, which is a wire electrically connected to a drain electrode of the memory cell.

[0017] Memory access operations can begin with an activate (ACT) command that selects a memory bank and a row, and copies data from the memory cells of the selected row to a row buffer for subsequent access. A read (RD) command, which can be issued after the activate command, specifies a starting column (bit line) position for a read operation, and causes data to be moved from the specified column position of the row buffer to the memory controller. A write (WR) command, which can be issued after the activate command, specifies a starting column (bit line) position for a write operation, and causes data supplied by the memory controller to be stored at the specified column position of the row buffer.

[0018] The selected row remains active (open) for access until a precharge (PRE) command is issued to the bank, which causes the open row to be closed by moving data from the row buffer to the memory cells of the open row. Alternatively, instead of issuing an explicit precharge command to deactivate a particular bank, an auto-precharge read (RDA) and an auto-precharge write (WRA) command can be used to automatically precharge the row after a read or write operation is completed. Once a bank has been precharged, it is in an idle state and should be activated before any read or write command is issued to the bank.

[0019] Accordingly, the row buffer efficiently caches data from the selected row. However, the efficiency of the caching scheme is limited in practice by the size of the cache memory, which in the above scenario is the single row buffer per memory bank. The systems and methods of the present disclosure improve memory access and overcome the above and other deficiencies of various memory device implementations by providing a dedicated on-die low-latency memory for caching data from main memory (e.g., from a DRAM memory bank). In some implementations, the on-die low-latency memory to be used as cache memory can be provided by static random access (SRAM) memory, which has an access latency at least an order of magnitude shorter than that of DRAM.

[0020] While the example implementations described herein use DRAM as main memory and SRAM as cache memory, the systems and methods of the present disclosure are not limited to or by any particular memory technology. Accordingly, advantages of systems and methods implemented in accordance with some embodiments of the present disclosure include, but are not limited to, improving memory access efficiency by providing low-latency on-die cache memory to be used for caching data from memory banks of main memory.

[0021] Figures 1A-1B A high-level component diagram of an example memory subsystem implemented in accordance with aspects of the present disclosure is schematically shown. As shown, the memory subsystem 100 includes one or more memory devices 110 coupled to a memory controller 120, which can be in communication with a host computer system (not shown) in accordance with some embodiments. The memory controller 120 and / or various other functional components of the memory subsystem 100 implement processing logic for performing the memory access operations described below. “Coupled to” in this context refers to an electrical connection between components, including indirect connections via one or more intervening components and direct connections (i.e., without intervening components). Figures 1A-1B Figures 1A-1B The memory controller 120 and / or various other functional components of the memory subsystem 100 implement processing logic for performing the memory access operations described below. “Coupled to” in this context refers to an electrical connection between components, including indirect connections via one or more intervening components and direct connections (i.e., without intervening components).

[0022] The memory device (e.g., DRAM device) 110 includes a plurality of memory banks 130A-130N, which are grouped into bank groups 140A-140K. While sixteen memory banks grouped into four bank groups are shown, various other implementations can employ other numbers of bank groups and banks. Each memory bank 130 is a memory array including a plurality of memory cells, such that each memory cell is capable of storing information for one or more bits depending on the memory cell type. Figures 1A-1B

[0023] ​​As mentioned above, the memory device 110 may further include a set of row buffers 150A-150L for storing data retrieved from rows of memory bank 130. In an illustrative example, one or more row buffers 150 may be dedicated to each memory bank 130. In another illustrative example, one or more row buffers 150 may be shared among several co-located memory banks 130 (e.g., among the banks of a single memory bank group 140). In yet another illustrative example, one or more row buffers 150 may be shared among all memory banks 130 of the memory device 110.

[0024] The memory device may further include an on-die cache memory 160, which can be used to cache portions of the data stored in the main memory banks 130A-130N. In an illustrative example, data already read from the memory banks into a row buffer can also be cached by the on-die cache memory 160, which can therefore be used to serve subsequent memory access requests routed to the same row. In some embodiments, the cache line size of the on-die cache memory 160 can be matched to the row buffer size, thereby simplifying the cache line allocation scheme that can be used to manage the cache memory 160.

[0025] For clarity and brevity, from Figures 1A-1B Various other components, such as sense amplifiers, input / output interfaces, and command interfaces, are omitted. In one embodiment, the memory device 110 may be implemented as one or more integrated circuits located on one or more dies. In another embodiment, the memory subsystem 100 may be implemented as a system-on-a-chip, wherein the memory subsystem, in addition to Figures 1A-1B In addition to the memory device 110 and the memory controller 120, it may also include one or more processing cores and one or more input / output (I / O) interfaces.

[0026] In some implementations, management of the cache memory 160 can be performed by circuitry located on the memory device 110. In an illustrative example, the allocation of cache lines in the cache memory 160 for storing data read from the memory bank can be managed by content-addressable memory (CAM) circuitry 170 on a die, such as... Figure 1A As illustrated schematically, CAM circuit 170 can decode the memory address specified by the memory access request into a reference to a line in cache memory 160—if the requested data has been previously cached—or into a reference to a memory bank containing the requested data and a row within that memory bank, without requiring any changes to the memory interface.

[0027] Alternatively, management of the cache memory 160 can be performed by a circuit system located external to the memory device 110. In an illustrative example, the CAM circuitry 170 for memory address translation can be located on a system-on-a-chip implementing the memory subsystem 100, such as... Figure 1B As shown schematically. Similar to Figure 1A In one implementation, CAM circuit 170 can decode the memory address specified in the memory access request into a reference to a line in cache memory 160—if the requested data has been previously cached—or into a reference to a memory bank containing the requested data and a row within that memory bank.

[0028] From memory device 110, system on chip 100 or host ( Figures 1A-1B A cache management strategy (not shown) may include cache rules and eviction rules. The cache rules specify which data should be cached, and the eviction rules specify which cache line should be evicted when no cache line is available to store new data. In some implementations, a baseline cache rule may specify that any new incoming data should be cached. Furthermore, one or more cache rules may implement a specific probing method relative to which data should be cached (or not cached). Such probing rules may specify logical conditions for evaluating data patterns, address ranges, etc. In an illustrative example, a cache rule may specify one or more memory address ranges that should be cached. In another illustrative example, a cache rule may specify one or more memory address ranges that should not be cached. In yet another illustrative example, a cache rule may specify a data pattern such that incoming data matching the pattern should be cached. In yet another illustrative example, a cache rule may specify a data pattern such that incoming data matching the pattern should not be cached.

[0029] Cache management policies may further include one or more eviction rules. In various illustrative examples, eviction rules may specify cache lines that should be evicted when no cache lines are available to store new data (e.g., First-In-First-Out (FIFO), Last-In-First-Out (LIFO), Least Recently Used, Least Frequently Used, Random Replacement, etc.). In some implementations, cache eviction rules may specify logical conditions for evaluating data patterns, address ranges, etc.

[0030] As mentioned above, each line of the cache memory may be associated with cache line metadata for a specified memory address (e.g., bank group identifier, memory bank identifier, and row address), the contents of which are currently stored in the cache line. In some embodiments, the cache line metadata associated with each line of the cache memory may further include a line status flag indicating whether the cache line has been modified since it was copied from main memory. Therefore, if the line status flag indicates that the cache line has been modified since it was copied from main memory, only the data stored in the cache line can be written back to main memory. Conversely, unmodified data in the cache line can be discarded after it is evicted.

[0031] exist Figure 1B The example implementations and other implementations in which the cache memory 160 is managed by a circuitry system located outside the memory device 110 can enhance the memory interface to facilitate signaling indicating the location of desired data in the main memory or cache memory.

[0032] In some implementations, memory device 110 may implement a flat address space for main memory bank 130 and cache memory 160. Therefore, address bits (e.g., least significant bit, most significant bit, or any other bit having a specified location within the address) may be selected as selectors between main memory bank 130 and cache memory 160, such that each of the two possible values ​​of the selected address bit can select either main memory bank 130 or cache memory 160.

[0033] Alternatively, signaling indicating the location of desired data in main memory or cache memory can be executed by reassigning certain command bits, such that the bit with the specified location within the command will be used as a selector between main memory bank 130 and cache memory 160. In various illustrative examples, activation, read, write, and / or precharge commands may be implemented accordingly.

[0034] In some implementations, the memory interface can be enhanced by implementing specific cache-related hints in several commands, thereby allowing the host to instruct the memory device to cache or evict specific data. In an illustrative example, the Activate (ACT) command can be enhanced to include cache hints, instructing the memory device to cache or not cache the contents of the activated word line, such as... Figure 2As illustrated, in this illustrative example, the baseline caching strategy implemented by the memory device may involve caching all data fetched from main memory unless the Activation (ACT) command provides a "Do not cache" prompt. In another illustrative example, if the Activation (ACT) command provides a "Cache" prompt, the baseline caching strategy implemented by the memory device may involve caching only the data fetched from main memory.

[0035] In some implementations, the memory interface can be enhanced by implementing commands and / or prompts that specify a range to be cached or not cached, thereby allowing the host instruction memory device to cache a specific address range or not cache a specific address range.

[0036] Figure 2 The interaction between host 210 and memory subsystem 215, executing an activation (ACT) command enhanced by cache hints according to various aspects of this disclosure, is illustrated schematically. As mentioned above, the activation command copies data from a memory cell in a selected row to a designated row buffer for subsequent access by a read or write command. In this illustrative example, host 210 may initiate a memory access operation by issuing an activation (ACT) command 220. The parameters of the activation command 220 may include a bank group address 230, a bank address 232, a row address 234, and a cache hint 236. Each of these parameters may be encoded by a bit group transmitted via a corresponding input from memory device 250.

[0037] Processing the activation command 220 may involve decoding the physical address fields 230, 232, and 234 via the row decoder 255, moving data from the selected row 260 (identified by row address 234) of the bank 265 (identified by bank address 232) of the bank group identified by bank group address 230 to the row buffer 270, and caching the data read from row 260 according to the baseline cache policy and the value of cache hint 236. In an illustrative example, if the baseline cache policy implemented by the memory device involves caching all data fetched from main memory, processing the activation command 220 will involve storing the data already read from row 260 into an available row of the cache memory 160, unless the activation (ACT) command 220 provides a "do not cache" hint. Conversely, if the baseline caching strategy implemented by the memory device involves only selective caching of data fetched from main memory, then when the activation (ACT) command 220 provides a "caching" prompt, processing the activation command 220 will involve storing the data read from the selected line 260 into an available line of the cache memory 160. The cache metadata of the cache line is then modified accordingly to associate the cache line with the address in main memory specified by the activation command 220. The cached data can then be used to serve subsequent memory access requests (read or write commands).

[0038] In some implementations, the precharge command can be enhanced to include an evict hint for the instruction memory device to evict the contents of a cache line storing a word line that is currently precharged, such as... Figure 3 The diagram schematically illustrates the interaction between host 210 and memory subsystem 215 executing an eviction prompt enhanced precharge (PRE) command according to various aspects of this disclosure. As mentioned above, host 210 can write the contents of a specified row buffer to memory device 250 by issuing a precharge (PRE) command 230. Therefore, the parameters of precharge command 230 may include bank address 230, bank address 232, row address 234, and eviction prompt 238. Each of these parameters may be encoded by a bit group transmitted via a corresponding input to memory device 250.

[0039] Processing the precharge command 230 may involve identifying the cache line associated with the line identified by bank set address 230, bank address 232, and line address 234. Data from the identified cache line may then be copied to the line buffer 270 associated with the line identified by bank set address 230, bank address 232, and line address 234, and / or copied to the word line identified by bank set address 230, bank address 232, and line address 234. If an eviction hint 238 is set, the cache line can be evicted, and the cache metadata can be modified accordingly.

[0040] Alternatively, the automatic precharge mode can be enhanced by using automatic precharge read (RDA) and automatic precharge write (WRA) commands to include a "perform eviction" prompt, whereby the instruction memory device evicts cache lines storing the contents of the word lines being read or written, respectively, such as... Figures 4-5 As shown schematically.

[0041] Specifically, Figure 4 The interaction between host 210 and memory subsystem 215, which executes an automatic precharge read (RDA) command enhanced with an evict prompt according to various aspects of this disclosure, is illustrated schematically. In this illustrative example, after completing activation command 220, host 210 can perform a memory access operation by issuing an automatic precharge read (RDA) command 235. The parameters of the automatic precharge read (RDA) command 235 may include bank group address 230, bank address 232, row address 234, column address 237, and evict prompt 238. Each of these parameters may be encoded by a bit group transmitted via a corresponding input from memory device 250.

[0042] The processing using the Automatic Precharge Read (RDA) command 235 may involve identifying a cache line associated with a specified memory address, and identifying a row buffer 270 associated with the specified memory address in response to a cache miss detection. Data from the identified cache line or row buffer is then copied to a data input / output buffer 280, starting from the bit line identified by column address 238, and the data is then transferred from the data input / output buffer to the memory controller. If an evictment hint 238 is set, the cache line can be evicted, and the cache metadata can be modified accordingly.

[0043] Figure 5The interaction between host 210 and memory subsystem 215, which execute an Automatic Precharge Write (WRA) command according to various aspects of this disclosure, is illustrated schematically, wherein line buffer allocation is managed by memory subsystem 215. In the illustrative example, after completing activation command 221, host 210 can perform a memory access operation by issuing Automatic Precharge Write (WRA) command 245.

[0044] The parameters of the Automatic Precharge Write (WRA) command 245 may include a bank address 230, a bank address 232, a row address 234, a column address 237, data 240 to be written to a specified location in the memory device 250, and an egress prompt 238. Each of these parameters may be encoded by a bit group transmitted via a corresponding input from the memory device 250.

[0045] The processing using the Automatic Precharge Write (WRA) command 245 may involve identifying a cache line associated with a specified memory address, and identifying a row buffer 270 associated with the specified memory address in response to a cache miss detection. Starting from the bit line identified by column address 238, the data item 240 supplied by the Automatic Precharge Write (WRA) command 245 is then copied to the identified cache line or row buffer 270. The contents of the cache line or row buffer 270 are then copied to the word line identified by bank set address 230, bank address 232, and row address 234. If an evictment hint 238 is set, the cache line is evicted, and the cache metadata is modified accordingly.

[0046] In some implementations, the memory interface can be enhanced to implement explicit "store in cache line" and "eject from cache line" commands. A "store in cache line" command explicitly instructs the memory device to copy data from a specified location in main memory to a specified cache line; conversely, a "eject from cache line" command instructs the memory device to copy data from a specified line in cache memory to a specified location in main memory.

[0047] In various embodiments of this disclosure, memory access requests served from cache memory 160 (i.e., if a cache hit occurs) can exhibit significantly lower access latency (e.g., an order of magnitude lower) than memory access requests served from main memory bank 130 in response to a cache miss detection. Therefore, memory subsystem implementations where the host is unaware of the underlying cache architecture can further enhance the memory interface to provide differentiated latency signaling to the host (e.g., by driving specific signals of the interface bus to a known state, thereby informing the host of the high or low latency involved in serving the current memory access request).

[0048] Figure 6 This is a flowchart of an example method 600 for performing memory access operations on a memory device operating according to embodiments of the present disclosure. As mentioned above, the memory device may include a plurality of memory banks grouped into several memory banks. The memory device may further include a plurality of row buffers. In an illustrative example, a subset of row buffers comprising one or more row buffers may be dedicated to each memory bank. In another illustrative example, a subset of row buffers comprising two or more row buffers may be shared among several co-located memory banks or shared among all memory banks of a memory subsystem. The memory device may further include cache memory. In some embodiments, the size of the cache line may be equal to the size of the row buffer. In some embodiments, the memory device is a dynamic random access memory (DRAM) device, and the cache memory is a static random access memory (SRAM) device.

[0049] As mentioned above, in some embodiments, the memory interface can be enhanced by implementing specific cache-related hints in several commands, thereby allowing the host to instruct the memory device to cache or evict specific data. Therefore, method 600 can be... Figures 2-5 The method is executed by the memory subsystem and / or memory device. Although the operations of the method are shown in a specific sequence or order, the order of operations may be modified unless otherwise specified. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated operations may be performed in different orders, and some operations may be performed in parallel. In addition, in some embodiments, one or more operations may be omitted. Therefore, not all of the illustrated operations are required in every embodiment, and other processing flows are possible.

[0050] In operation 610, the memory device implementing the method receives an activation command enhanced by cache hints. The parameters of the activation command include a bank set address, a bank address, a line address, and cache hints. Each of these parameters can be encoded via a bit group transmitted through a corresponding input of the memory device.

[0051] In operation 615, the memory device fetches data from the row specified by the parameters of the activation command into the row buffer.

[0052] In operation 620, the memory device copies data to cache lines based on cache hints and cache policies associated with the memory device, the cache lines being identified by CAM circuitry that translates the memory address specified by the parameters of the activation command. If no cache line is previously associated with the specified address, an available cache line can be associated with the specified address, and the cache metadata can be updated accordingly. If no cache line is available, a cache line can be evicted according to an eviction policy implemented by the memory device.

[0053] In an illustrative example, if the caching strategy implemented by the memory device involves caching all data fetched from main memory, then processing the activation command will involve caching the data already read from the specified line, unless the activation command provides a "do not cache" prompt. Conversely, if the caching strategy implemented by the memory device only involves selectively caching data fetched from main memory, then processing the activation command will involve caching the data read from the specified line if the activation command provides a "cache" prompt.

[0054] In operation 625, the memory device receives a read command for a specified column address.

[0055] In operation 630, the memory device reads data from the cache line with an offset identified by the column address.

[0056] In operation 635, the memory device receives a write command specifying the column address to be stored on the memory device and the new data item.

[0057] In operation 640, the memory device stores new data items into the cache line with an offset identified by the column address.

[0058] In operation 645, the memory device receives a precharge command. The parameters of the precharge command include the bank group address, the bank address, the row address, and an evictment hint. Each of these parameters can be encoded using a bit group transmitted via the corresponding input of the memory device.

[0059] In operation 650, the memory device copies data from the cache line to the memory bank location specified by the command parameters.

[0060] In operation 655, the memory device evicts cache lines based on cache hints and cache policies associated with the memory device. In an illustrative example, if an eviction hint supplied by a precharge command is set, the cache line is evicted, and the cache metadata may be modified accordingly. The method terminates after operation 655 is completed.

[0061] Figure 7An example computing system 700 is illustrated, comprising a memory subsystem 710 implemented according to some embodiments of the present disclosure. The memory subsystem 710 may include media, such as one or more volatile memory devices (e.g., memory device 740), one or more non-volatile memory devices (e.g., memory device 730), or a combination thereof. In some embodiments, the memory subsystem 710 corresponds to... Figures 1A-1B The memory subsystem 100.

[0062] The memory subsystem 710 can be a storage device, a memory module, or a combination of both. Examples of storage devices include solid-state drives (SSDs), flash drives, universal serial bus (USB) flash drives, embedded multimedia controller (eMMC) drives, universal flash memory (UFS) drives, secure digital cards (SD cards), and hard disk drives (HDDs). Examples of memory modules include dual in-line memory modules (DIMMs), small form factor DIMMs (SO-DIMMs), and various types of non-volatile dual in-line memory modules (NVDIMMs).

[0063] Computer system 700 may be a computing device such as a desktop computer, laptop computer, web server, mobile device, vehicle (e.g., airplane, drone, train, car or other means of transport), Internet of Things (IoT) enabled device, embedded computer (e.g., computer contained in a vehicle, industrial equipment or networked commercial device), or such computing device containing memory and processing means (e.g., processor).

[0064] Computer system 700 may include host system 720 coupled to one or more memory subsystems 710. In some embodiments, host system 720 is coupled to different types of memory subsystems 710. Figure 7 An example of a host system 720 coupled to a memory subsystem 710 is shown. The host system 720 may include a processor chipset and a software stack executed by the processor chipset. The processor chipset may include one or more cores, one or more cache memories, a memory controller (e.g., an NVDIMM controller), and a storage protocol controller (e.g., a PCIe controller, a SATA controller). For example, the host system 720 uses the memory subsystem 710 to write data to and read data from the memory subsystem 710.

[0065] The host system 720 can be coupled to the memory subsystem 710 via a physical host interface. Examples of physical host interfaces include, but are not limited to, Serial Advanced Technology Attachment (SATA) interfaces, Peripheral Component Interconnect High Speed ​​(PCIe) interfaces, Universal Serial Bus (USB) interfaces, Fibre Channel, Serial Attached SCSI (SAS), Dual Data Rate (DDR) memory bus, Small Computer System Interface (SCSI), Dual In-line Memory Module (DIMM) interfaces (e.g., DIMM sockets supporting Dual Data Rate (DDR)), Open NAND Flash Interface (ONFI), Dual Data Rate (DDR), Low Power Dual Data Rate (LPDDR), etc. The physical host interface can be used to transfer data between the host system 720 and the memory subsystem 710. When the memory subsystem 710 is coupled to the host system 720 via the PCIe interface 105, the host system 720 can further utilize the NVM High Speed ​​(NVMe) interface to access components (e.g., memory device 730). The physical host interface 105 provides an interface for transmitting control, address, data and other signals between the memory subsystem 710 and the host system 720. Figure 7 Memory subsystem 710 is shown as an example. Generally, host system 720 can access multiple memory subsystems via the same communication connection, multiple separate communication connections, and / or combinations of communication connections.

[0066] Memory devices 730 and 740 may comprise any combination of different types of non-volatile memory devices and / or volatile memory devices. Volatile memory devices (e.g., memory device 740) may be, but are not limited to, random access memory (RAM), such as dynamic random access memory (DRAM) and synchronous dynamic random access memory (SDRAM).

[0067] Some examples of non-volatile memory devices (e.g., memory device 730) include NAND flash memory and in-place write memory, such as three-dimensional crosspoint (“3D crosspoint”) memory devices, which are crosspoint arrays of non-volatile memory cells. The crosspoint array of non-volatile memory can perform bit storage based on variations in volume resistance in conjunction with a stackable cross-grid data access array. Furthermore, compared to many flash-based memories, crosspoint non-volatile memory can perform in-place write operations, where non-volatile memory cells can be programmed without pre-erasing them. NAND flash memory includes, for example, two-dimensional NAND (2D NAND) and three-dimensional NAND (3D NAND).

[0068] Each of the memory devices 730 may include one or more arrays of memory cells. One type of memory cell, such as a single-level cell (SLC), may store one bit per cell. Other types of memory cells, such as multi-level cell (MLC), three-level cell (TLC), and four-level cell (QLC), may store multiple bits per cell. In some embodiments, each of the memory devices 730 may include one or more arrays of memory cells, such as SLC, MLC, TLC, QLC, or any combination thereof. In some embodiments, a particular memory device may include an SLC portion of memory cells, as well as an MLC portion, a TLC portion, or a QLC portion. The memory cells of the memory device 730 may be grouped into pages, which may refer to logical cells of the memory device used for storing data. For some types of memory (e.g., NAND), pages may be grouped to form blocks.

[0069] Although non-volatile memory devices, such as arrays of 3D cross-point non-volatile memory cells and NAND-type memories (e.g., 2D NAND, 3D NAND), are described, memory device 730 may be based on any other type of non-volatile memory, such as read-only memory (ROM), phase-change memory (PCM), auto-select memory, other chalcogenide-based memories, ferroelectric transistor random access memory (FeTRAM), ferroelectric random access memory (FeRAM), magnetic random access memory (MRAM), spin-transfer torque (STT)-MRAM, conductive bridged RAM (CBRAM), resistive random access memory (RRAM), oxide-based RRAM (OxRAM), NOR flash memory, and electrically erasable programmable read-only memory (EEPROM).

[0070] The memory subsystem controller 775 can communicate with the memory device 730 to perform operations such as reading data, writing data, or erasing data at the memory device 730, and other such operations. The memory subsystem controller 775 may include hardware, such as one or more integrated circuits and / or discrete components, buffer memories, or combinations thereof. The hardware may include a digital circuit system having dedicated (i.e., hard-decoded) logic that performs the operations described herein. The memory subsystem controller 775 may be a microcontroller, a dedicated logic circuit system (e.g., a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc.), or other suitable processor.

[0071] The memory subsystem controller 775 may include a processor 717 (e.g., a processing device) configured to execute instructions stored in local memory 719. In the illustrated example, the local memory 719 of the memory subsystem controller 775 includes embedded memory configured to store instructions for performing various processes, operations, logic flows, and routines that control the operation of the memory subsystem 710, including handling communication between the memory subsystem 710 and the host system 720. In some embodiments, the processor 717 may implement methods 600-650, as described in more detail above.

[0072] In some embodiments, local memory 719 may include memory registers storing memory pointers, retrieved data, etc. Local memory 719 may also include read-only memory (ROM) for storing microcode. Although Figure 7 The example memory subsystem 710 has been shown to include a controller 775, but in another embodiment of this disclosure, the memory subsystem 710 does not include a controller 775 and may instead rely on external control (e.g., provided by an external host or by a processor or controller separate from the memory subsystem).

[0073] Generally, the memory subsystem controller 775 can receive commands or operations from the host system 720 and can translate these commands or operations into instructions or appropriate commands to enable the desired access to the memory device 730. The memory subsystem controller 775 may handle other operations such as wear leveling, garbage collection, error detection and error correction (ECC) operations, encryption, caching, and address translation between logical addresses (e.g., logical block addresses, namespaces) and physical addresses (e.g., physical block addresses) associated with the memory device 730. The memory subsystem controller 775 may further include a host interface circuitry for communicating with the host system 720 via a physical host interface. The host interface circuitry can translate commands received from the host system into command instructions to access the memory device 730 and translate responses associated with the memory device 730 into information for the host system 720.

[0074] The memory subsystem 710 may also include additional circuitry or components not shown. In some embodiments, the memory subsystem 710 may include a cache memory or buffer (e.g., DRAM) and an address circuitry (e.g., a row decoder and a column decoder) that can receive and decode addresses from the controller 775 to access the memory device 730.

[0075] In some embodiments, memory device 730 includes a local media controller 735 that operates together with memory subsystem controller 775 to perform operations on one or more memory cells of memory device 730. An external controller (e.g., memory subsystem controller 775) may externally manage memory device 730 (e.g., perform media management operations on memory device 730). In some embodiments, memory subsystem 710 is a managed memory device, which is the original memory device 730 having on-die control logic (e.g., local media controller 735) and a controller (e.g., memory subsystem controller 775) for media management within the same memory device package. An example of a managed memory device is a managed NAND (MNAND) device.

[0076] Figure 8 An example machine of computer system 800 is shown, within which a set of instructions for causing the machine to perform any one or more methods discussed herein is executable. In some embodiments, computer system 800 may correspond to a host system (e.g., Figure 7 The host system 120 includes, is coupled to, or utilizes a memory subsystem (e.g., Figures 1A-1B The memory subsystem 100 or Figure 7 (Memory subsystem 710).

[0077] In alternative embodiments, the machine may be connected (e.g., coupled to network interface device 838 of network 820) to other computer systems in a LAN, corporate intranet, extranet, and / or the Internet. The machine may operate as a peer machine in a peer-to-peer (or distributed) network environment or as a server or client machine in a cloud computing infrastructure or environment, or within the capacity of a server or client machine in a client-server network environment.

[0078] The machine may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), cellular phone, network appliance, server, network router, switch, or bridge, or any machine capable of (sequentially or otherwise) executing a set of instructions specifying actions to be taken by the machine. Furthermore, although a single machine is shown, it should also be understood that the term "machine" includes any set of machines that individually or collectively execute a set of instructions (or multiple sets of instructions) to perform any one or more methods discussed herein.

[0079] Example computer system 800 includes a processing device 802, a main memory 804 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM), such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), etc.), a static memory 808 (e.g., flash memory, static random access memory (SRAM), etc.), and a data storage system 818, which communicate with each other via a bus 830.

[0080] Processing device 802 represents one or more general-purpose processing devices, such as microprocessors, CPUs, etc. More specifically, the processing device may be a Complex Instruction Set Computing (CISC) microprocessor, a Reduced Instruction Set Computing (RISC) microprocessor, a Very Long Instruction Word (VLIW) microprocessor, or a processor implementing other instruction sets or a combination of instruction sets. Processing device 802 may also be one or more special-purpose processing devices, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), network processors, etc. Processing device 802 is configured to execute instructions 828 for performing the methods discussed herein (e.g., managing on-die cache memory, as described in more detail above).

[0081] Data storage system 818 may include machine-readable storage medium 824 (also referred to as computer-readable medium) on which one or more sets of instructions 828 or software embodying any one or more methods or functions described herein are stored. Instructions 828 may also reside wholly or at least partially within main memory 804 and / or processing device 802 during execution by computer system 800, which also constitute machine-readable storage medium. Machine-readable storage medium 824, data storage system 818, and / or main memory 804 may correspond to... Figure 7 The memory subsystem 110.

[0082] In one embodiment, according to some embodiments of this disclosure, instruction 828 includes instruction 834 for implementing and managing a device-on-die cache memory. Although machine-readable storage medium 824 is shown as a single medium in the example embodiment, the term "machine-readable storage medium" should be considered to include a single medium or multiple media storing one or more sets of instructions. The term "machine-readable storage medium" should also be considered to include any medium capable of storing, encoding, or causing a machine to perform any one or more methods of this disclosure. Therefore, the term "machine-readable storage medium" should be considered to include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0083] Some parts of the previously described descriptions have been presented based on the algorithms and symbolic representations of operations on data bits within computer memory. These algorithmic descriptions and representations are the means by which those skilled in the art of data processing most effectively communicate the essence of their work to others skilled in the art. Algorithms are, and are generally considered, a self-consistent sequence of operations that produce the desired result. Operations are those that require the physical manipulation of physical quantities. These quantities are usually, but not necessarily, in the form of electrical or magnetic signals that can be stored, combined, compared, and otherwise manipulated. Sometimes, primarily for general reasons, it has proven convenient to refer to these signals as bits, values, elements, symbols, characters, terms, quantities, etc.

[0084] However, it should be remembered that all these and similar terms should be associated with appropriate physical quantities and are merely convenient notations applied to those quantities. This disclosure can refer to the actions and processes of a computer system or similar electronic computing device that manipulate data represented as physical (electronic) quantities in the registers and memories of the computer system and transform said data into other data similarly represented as physical quantities in the memory or registers or other such information storage systems of the computer system.

[0085] This disclosure also relates to apparatus for performing the operations described herein. Such apparatus may be specifically constructed for its intended purpose, or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in a computer. This computer program may be stored in a computer-readable storage medium, such as, but not limited to, any type of disk, including floppy disks, optical disks, CD-ROMs and magneto-optical disks, read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic cards or optical cards, or any type of media suitable for storing electronic instructions, each coupled to a computer system bus.

[0086] The algorithms and displays presented herein are not inherently related to any particular computer or other device. Various general-purpose systems can be used with the programs taught herein, or it may be convenient to construct more specialized devices to perform the methods described herein. The structures of various such systems will be presented as set forth in the description below. Furthermore, this disclosure is described without reference to any particular programming language. It should be understood that the teachings of this disclosure as described herein can be implemented using various programming languages.

[0087] This disclosure can be provided as a computer program product or software, which may include a machine-readable medium having instructions stored thereon for programming a computer system (or other electronic device) to perform processes according to this disclosure. The machine-readable medium includes any mechanism for storing information in a machine-readable (e.g., computer-readable) form. In some embodiments, the machine-readable (e.g., computer-readable) medium includes machine-readable storage media, such as read-only memory (“ROM”), random access memory (“RAM”), disk storage media, optical storage media, flash memory devices, etc.

[0088] In the foregoing description, embodiments of the present disclosure have been described with reference to specific exemplary embodiments thereof. It will be apparent that various modifications may be made thereto without departing from the broader spirit and scope of the embodiments of the present disclosure as set forth in the appended claims. Therefore, the description and drawings should be regarded as illustrative rather than restrictive.

Claims

1. A memory device comprising: Multiple memory bank groups, wherein each memory bank group includes multiple memory banks; Multiple row buffers, wherein at least one of the multiple row buffers is associated with each of the multiple memory banks; A cache memory includes a plurality of cache lines, wherein the size of each cache line is equal to the size of a first row buffer among the plurality of row buffers; Processing logic, communicatively coupled to the plurality of memory banks and the plurality of row buffers, is configured to perform operations including the following: Receive an activation command, the activation command including a cache hint and an identifier of a row of a memory bank among the plurality of memory banks, wherein the cache hint is encoded by one or more bits of the activation command and indicates whether the memory device caches or excludes the contents of the identified row. Extract data from the row identified by the activation command into the second row buffer among the plurality of row buffers; as well as Based on the cache hint and a caching policy specifying selective data caching, the data is copied to a cache line among the plurality of cache lines, wherein the caching policy specifies one of the following: (i) caching the data unless the cache hint specifies exclusion from caching, or (ii) caching the data only when the cache hint specifies caching.

2. The memory device according to claim 1, wherein the memory device is a dynamic random access memory (DRAM) device.

3. The memory device according to claim 1, wherein the cache memory is a static random access memory (SRAM) device.

4. The memory device of claim 1, wherein the processing logic includes content-addressable memory (CAM) circuitry to translate a memory address into one of the following: an identifier of a row of a memory bank among the plurality of memory banks or an identifier of a line of the cache memory.

5. The memory device of claim 1, wherein the processing logic is configured to further perform operations including: Receives a read command for a specified column address; At least a portion of the data is read from the location of the cache line, wherein the location is identified by the column address.

6. The memory device of claim 1, wherein the processing logic is configured to further perform operations including the following: Receives a write command with a specified column address and new data; The new data is stored at the location of the cache line, wherein the location is identified by the column address.

7. The memory device of claim 1, wherein the processing logic is configured to further perform operations including: Receive a precharge command specifying the row of the memory bank; The data is copied from the cache line to the row of the memory bank.

8. A system-on-a-chip, comprising: Multiple memory bank groups, wherein each memory bank group includes multiple memory banks; Multiple row buffers, which are communicatively coupled to the multiple memory banks; A cache memory includes a plurality of cache lines, wherein the size of each cache line is equal to the size of a row buffer among a plurality of row buffers; Content-Addressable Memory (CAM) circuitry is used to translate a specified memory address into one of the following: an identifier of a row of a memory bank among the plurality of memory banks or an identifier of a line of the cache memory. Processing logic, communicatively coupled to the plurality of memory banks and the plurality of row buffers, is configured to perform operations including the following: Implement a caching policy that specifies selective data caching, wherein the caching policy specifies one of the following: (i) cache the data unless the cache prompts to exclude it from caching, or (ii) cache the data only when the cache prompts to exclude it from caching; Receive a read command for a specified memory address, the memory address including a cache selector bit located at a specified position within the memory address, wherein a first value of the cache selector bit indicates that the memory address references a memory location, and a second value of the cache selector bit indicates that the memory address references a cache location; Based on the cache selector bit, the memory address is translated into the identifier of the cache line among the plurality of cache lines by the CAM circuit; as well as Data is read from the cache line, wherein the cache policy specifies logical conditions for evaluating data patterns or memory address ranges.

9. The system-on-a-chip according to claim 8, wherein each of the plurality of memory banks is a dynamic random access memory (DRAM) bank.

10. The system-on-a-chip according to claim 8, wherein the cache memory is a static random access memory (SRAM) device.

11. The system-on-a-chip of claim 8, wherein one or more of the plurality of row buffers are associated with each of the plurality of memory banks.

12. The system-on-a-chip of claim 8, wherein the processing logic is configured to further perform operations including the following: Receive a write command specifying the memory address and new data; The memory address is translated into the identifier of the cache line among the plurality of cache lines by the CAM circuit; and The new data is stored in the cache line.

13. The system-on-a-chip according to claim 8, wherein the processing logic is configured to further perform operations including the following: Receive a precharge command specifying the row of the memory bank; The data is copied from the cache line to the row of the memory bank.

14. A method comprising: The memory device receives an activation command, the activation command including a cache hint and an identifier of a row of a memory bank among a plurality of memory banks of the memory device, wherein the cache hint is encoded by one or more bits of the activation command and indicates whether the memory device caches or excludes the contents of the identified row. Data is extracted from the rows identified by the activation command into row buffers among multiple row buffers in the memory device; Based on the cache hint and a caching policy specifying selective data caching, the data is copied to a plurality of cache lines in the memory device, wherein the size of each of the plurality of cache lines is equal to the size of a row buffer in the plurality of row buffers, wherein the caching policy specifies one of the following: (i) caching the data unless the cache hint specifies exclusion from caching, or (ii) caching the data only when the cache hint specifies caching.

15. The method of claim 14, wherein the memory device is a dynamic random access memory (DRAM) device.

16. The method of claim 14, wherein the plurality of cache lines reside on a static random access memory (SRAM) device.

17. The method of claim 14, further comprising: The identifiers of the memory bank and the row are translated into the identifiers of the cache line using a content-addressable memory (CAM) circuit.

18. The method of claim 14, further comprising: Receives a read command for a specified column address; At least a portion of the data is read from the location of the cache line, wherein the location is identified by the column address.

19. The method of claim 14, further comprising: Receives a write command with a specified column address and new data; The new data is stored at the location of the cache line, wherein the location is identified by the column address.

20. The method of claim 14, further comprising: Receive a precharge command specifying the row and evictment prompt of the memory bank; The cache line is evicted in response to determining that the eviction prompt is set to a first value; as well as The data is copied from the cache line to the row of the memory bank.

Images