System applications for dram components with cache mode

Abstract

A memory system is disclosed, which has a memory controller and may also have memory components. The memory components may be dynamic random access memory (DRAM). The memory controller may be connected to the memory components. The memory components have at least one data row and at least one tag row, the at least one tag row being different from and associated with the at least one data row. The memory system is used to implement a cache with multiple paths to hold data groups. The memory controller can operate in each of a plurality of operating modes. These operating modes include a first operating mode and a second operating mode. The first operating mode and the second operating mode have different addressing and timing for accessing data groups. The memory controller has cache read logic for sending a cache read command, cache result logic for receiving a response from the memory components, and cache fetch logic.

Description

[0001] Citation of relevant applications

[0002] This application relates to and claims the benefit of U.S. Provisional Patent Application No. 62 / 820,144, entitled “SYSTEM APPLICATION OF DRAMCOMPONENT WITH CACHE MODE”, filed March 18, 2019, the entire contents of which are incorporated herein by reference. Background Technology

[0003] Dynamic Random Access Memory (DRAM) is widely used as system memory in a variety of computers and computing devices. Currently, DRAM expansion is slowing, and lower-cost Storage Class Memory (SCM) and DRAM will help continue to increase memory system capacity. Because SCM has longer latency and limited endurance compared to DRAM, hybrid memory systems combining SCM with DRAM caches may be useful. Attached Figure Description

[0004] This disclosure will be more fully understood in light of the detailed descriptions given below and the accompanying drawings, which illustrate various embodiments of this disclosure.

[0005] Figure 1A An embodiment of a DRAM with data rows and tag rows that can be operated as a multipath set-association cache or with a multipath set-association cache is illustrated.

[0006] Figure 1B The diagram illustrates what is suitable for Figure 1A The separate label in mat for the version of the DRAM embodiment.

[0007] Figure 2A The illustration shows another embodiment of DRAM with cached data distributed across multiple banks.

[0008] Figure 2B The diagram illustrates what is suitable for Figure 2A The data lines and tag lines of the DRAM embodiment in the example.

[0009] Figure 2C The diagram illustrates what is suitable for Figure 2A The distributed tag version of the DRAM embodiment in the example.

[0010] Figure 3A The illustration shows a circuit system and addressing for cache hits and access to data in a set of paths that are already cached in a multipath set associated cache, in one embodiment.

[0011] Figure 3BThe illustration shows how to use a set of bits from the tag data to access cached data pointed to by the matching tag in column decoding.

[0012] Figure 3C The illustration shows an SCM and the address space in DRAM for accessing data cached in a multipath set-association cache DRAM according to one embodiment.

[0013] Figure 4A The illustration shows the timing details of normal access to data and modified access to cached data in one embodiment.

[0014] Figure 4B The diagram illustrates the use of Figure 4A The read timing shown reads cached data from one memory bank of the multi-path set-associative cache DRAM, while simultaneously reading more cached data from other memory banks in parallel.

[0015] Figure 5A The illustration shows the timing details of normal access to data and modified access to cached data in one embodiment.

[0016] Figure 5B The diagram illustrates the use of Figure 5A The write timing shown writes cache data to one of the memory banks of the multipath set associated cache DRAM, while more cache data is written in parallel to other memory banks.

[0017] Figure 6 The illustration shows a DRAM cell and a sense amplifier in a mat, which can be used in an embodiment of DRAM.

[0018] Figure 7 The diagram illustrates the mode register used to select between standard DRAM mode and multipath set-associative cache mode with adjustable bit-length tags and comparators.

[0019] Figure 8A This is a flowchart illustrating the operation of a multipath set-associative DRAM cache, which can be practiced through the various embodiments described herein.

[0020] Figure 8B This is a flowchart illustrating the selection of operating modes that can be practiced through the various embodiments described herein.

[0021] Figure 9 An embodiment of a multipath set-associative DRAM cache is described.

[0022] Figure 10 Another embodiment of a multipath set-associative DRAM cache is described.

[0023] Figure 11 The present invention describes a DRAM according to this embodiment and a memory controller that can operate the DRAM as a multi-path set associated with a DRAM cache.

[0024] Figure 12 A controller using DRAM with a cache mode according to this embodiment is described.

[0025] Figure 13A Example XA of a system configuration in which data pathways are maintained in multiple storage bodies across multiple devices and tag access / data access overlaps is described.

[0026] Figure 13B Example YA of a system configuration in which data pathways are maintained in multiple storage bodies of a single device and tag access / data access overlaps.

[0027] Figure 13C ZA is a system configuration example in which data pathways are kept within a single storage body and tag access / data access overlaps.

[0028] Figure 13D ZB is a system configuration example in which data pathways are kept within a single storage body and tag access / data access do not overlap.

[0029] Figure 14A Describes the use of, for example Figure 13A The address field manipulation in the controller of Figure 12 shows the system configuration example XA.

[0030] Figure 14B Describes the use of, for example Figure 13B The system configuration example YA shown is Figure 12 Manipulation of the address field in the controller.

[0031] Figure 14C Describes the use of, for example Figure 13C The system configuration example ZA shown is Figure 12 Manipulation of the address field in the controller.

[0032] Figure 14D Describes the use of, for example Figure 13D The system configuration example shown is ZB. Figure 12 Manipulation of the address field in the controller.

[0033] Figure 15A Describes the use of, for example Figure 13D The system configuration example shown is the command format for ZB.

[0034] Figure 15B Describes the use of, for example Figure 13C , Figure 13B and Figure 13A The system configuration examples shown are the command formats for ZA, YA, and XA.

[0035] Figure 15C Describing as Figure 13D , Figure 13C , Figure 13B and Figure 13A The system configuration examples shown are the command formats for ZB, ZA, YA, and XA.

[0036] Figure 16 The timing details of reads with cache misses during fetching and evicting are described in DRAM with cache mode.

[0037] Figure 17 It provides the timing details for 64-byte reads, normal (direct) accesses, and modified (cached) accesses in DRAM with cache mode.

[0038] Figure 18 It is a flowchart depicting the controller sorting of DRAM caches that have experienced cache misses.

[0039] Figure 19 It describes the execution Figure 18 System action diagram of the controller using DRAM with cache mode. .

[0040] Figure 20 Touch markers are enhanced to depict the eviction decision-making process.

[0041] Figure 21A A 4kB cache line distributed across memory banks / channels / devices according to this embodiment is depicted.

[0042] Figure 21B The 4kB cache line tags and data of 16 sets (paths) located in the same DRAM row according to this embodiment are depicted. Detailed Implementation

[0043] In the following description, various aspects of the illustrative embodiments are described using terminology commonly used by those skilled in the art to convey the essence of their work to others skilled in the art. Specific numbers, materials, and configurations are stated for purposes of explanation to provide a thorough understanding of the illustrative embodiments. However, it will be apparent to those skilled in the art that this disclosure may be practiced only in some of the described aspects or without specific details. In other instances, well-known features have been omitted or simplified to avoid obscuring the illustrative embodiments.

[0044] The DRAM architecture and controller modifications described in this paper enable tag storage and comparison, as well as access to cached data, on the DRAM. In some versions, the DRAM can operate as a multi-path set-associative cache, and it can also operate as standard DRAM, for example, if it is not part of a hybrid system with storage-class memory or other memory. Sets of data from storage-class memory or other memory can be cached in individual paths within the DRAM cache and accessed when a tag comparison indicates a cache hit.

[0045] In a typical system, tag matching is performed via an external electrical path system outside the DRAM die. In hybrid memory systems, using a portion of standard DRAM as a cache involves transferring tag information bits from the DRAM to the chip so that tag matching can be performed before the data information on the DRAM can be accessed. As described herein, this incurs latency and power overhead compared to modifying the DRAM to enable tag matching.

[0046] Various embodiments of DRAM caches have some or all of the following features.

[0047] Typically, very large block sizes are used for transfers to and from storage-class memory (SCM). This results in a much smaller tag-to-data ratio than traditional caches, enabling DRAM modifications to the cache with minimal overhead. Refer to Figures 1 to 2. Figure 11 Various embodiments of DRAM having data rows and tag rows, as well as data mat rows and tag mat rows, are shown and described, and other embodiments having other arrangements of data rows, tag rows, data mat rows, and tag mat rows are readily designed for various tag-to-data ratios. Figures 12 to 21B The controller and DRAM of the multipath set associated cache are described in various configurations.

[0048] DRAM caches are organized as set-associative caches. In a set-associative cache, sets of data from memory-level storage can be cached in sets, in sets within any of multiple paths or locations, and in cache memory. The total number of paths in each set is multiplied by the total number of records in the cache memory to obtain the total number of lines in the cache memory. Each path and each set is mapped to a line. When a set of data is cached, a tag matches a portion of the set's address and points to the set and path or location in cache memory so that the tagged cached set can be read from the cache memory. This tag associates the set of data with the set and path or location in cache memory, hence the name of this type of cache. The number of data sets that can be cached, the number of sets, and the number of paths or locations that can be cached (typically denoted by N) determine the number of cache memories for an N-path set-associative cache and the total number of address bits used by each tag.

[0049] Tag storage in DRAM is accomplished by adding additional cell rows, either distributed within existing mats or added to them. A mat (whether a data mat with data rows, a tag mat with tag rows, or a mat with both data and tag rows) is a group of DRAM cells with defined height (number of cells or rows) and width (number of cells or columns). Each DRAM cell contains transistors (typically NMOS or N-type metal-oxide-semiconductor) and capacitors. [Reference] Figures 1A to 6 Various embodiments of mat are shown and described, and other arrangements of DRAM cells and mat are easy to design.

[0050] Associating a tag row with a data row that does not share a primary sense amplifier eliminates the need to double the sense amplifier stripe and saves area. In some embodiments, a tag row in a mat with one sense amplifier is associated with and points to a data location elsewhere in DRAM with another sense amplifier. This supports timing overlap in tag sensing through the sense amplifier operating for the tag row and data sensing through the sense amplifier operating for the data location associated with the tag row, as shown in the timing details of Figures 4 and 5. If the tag row and the associated data location are in the same mat, this timing overlap may require doubling the sense amplifier.

[0051] The labels of different pathways within the set are located in a row. A label-hit activation matches the label of one of the pathways located in that row, as shown in the reference. Figure 3A As shown and described. This feature supports linking together in a logical OR configuration for use Figure 3A The comparison blocks shown are for a compact circuit system, with each tag and tag matching a comparison block.

[0052] Tag comparison is performed solely through column access. All tags for a given set are located in the same row, and column accesses of tag data across that row are compared in parallel with address bits, such as... Figure 3A and Figure 3B As shown.

[0053] Parallel access at the edges of memory banks is used to maximize parallel tag comparisons. Tags are compared at the edges of each memory bank to allow multiple tag comparisons to be performed in parallel across multiple memory banks. In some embodiments, cached data is then accessed in parallel across multiple memory banks, for example, as... Figures 1A to 2C As shown.

[0054] Sets are addressed using device, bank, and row addresses. Data from storage-level memory can be cached in one of N paths or locations, in one set of cached sets, or in an N-path set-associative cache. Addresses used for sets include device addresses, bank addresses, and row addresses, as decoded via address decoding.

[0055] The set address, comparison value, and address determined by tag matching are distributed in DRAM using an internal address bus. See example. Figure 9 ,Should Figure 9 The cache logic and multiplexing used to distribute addresses to the memory banks in DRAM for row and column decoding are shown.

[0056] The internal data bus and DQ pins outside the array are used for data only. See, for example... Figure 9 ,Should Figure 9 This illustrates the internal data flow from the memory bank in DRAM to the data logic. Addresses and data are not mixed or multiplexed on the data bus.

[0057] Figure 1A The illustration shows an embodiment of a DRAM having data mat rows 104 (each data mat row has multiple data rows) and tag mat rows 106 (each tag mat row has multiple tag rows for tag information bits), which can operate as a multi-path set-associative cache. In this example, each memory bank 102 of the DRAM is an array of 32 mat 108 × 64 mat 108, and each mat 108 is a 512-bit × 512-bit array or subarray (see [link]). Figure 6Each mat 108 contains a total of 256 kilobits (kb). There are eight memory banks 102 in each channel A and B of DRAM 0, and eight more memory banks 102 in each channel C and D of DRAM 1, for a total of 32 memory banks 102. DRAM can perform 64-byte read or write accesses on 4 kilobits (kB) of DRAM blocks (cache lines). In this example, the labels in the label rows of the 32 mat 108s forming label mat rows 106 at the bottom (or top in other embodiments) of the memory bank 102 occupy approximately 1.6% of the additional memory cell area.

[0058] The DRAM memory bank 102 is constructed from a mat block array (subarray) containing memory cells and row / column peripheral logic. Data lines connect the memory bank 102 to the interface block. A single unmodified mat behavior label provides sufficient capacity and bandwidth (BW).

[0059] Figure 1B The diagram illustrates what is suitable for Figure 1A The DRAM embodiment in which the tag is individually labeled mat 112. The tag read bit determines whether cached data or tag data is read during a read operation. If the tag read bit is 1, the tag data in tag mat 112 is read. If the tag read bit is 0, cached data is read from data mat 110.

[0060] In various embodiments, including for such Figure 1A , Figure 1B , Figure 2A , Figure 2B and Figure 2C The tags shown can be read using normal access (if no tag matches, the row address to be activated is accessed directly from the memory controller). In some embodiments, it is necessary to signal the tag access to an additional address space that is not normally accessed. Internally, this can be done as for tags in a separate mat 112. Figure 1B The label reading bits are set as shown, or as... Figure 2B and Figure 2C The distribution shown is in the mat, etc. Several options can achieve this:

[0061] 1. When the DRAM is in cache mode, the bit allocation used to decode the bit sequence sent via the CA bus is modified to have an additional state that signals tag access.

[0062] 2. When the DRAM is in cache mode and there is a bidirectional signal such as DMI used to transmit additional information, this signal can be used to switch between cache access and tag reading.

[0063] 3. The Mode Register Set (MRS) switches between cache access and tag reading.

[0064] Figure 2A The illustration shows another embodiment of a DRAM where the cached data of a set is distributed across multiple memory banks 202. In this example, the set is distributed across four memory banks 202, each memory bank 202 being 32kB, and the per-path distribution of the set is 16 bytes per memory bank 202 in each of the four identical memory banks 202. Each of these four identical memory banks 202 has 32 tags. Other sets are distributed across other memory banks 202 in the DRAM, each set consisting of four memory banks 202. The tags in the tags of each of the four memory banks 202 occupied by the set match the cached data of the path pointing to the set in each of the four memory banks 202.

[0065] Figure 2B The diagram illustrates what is suitable for Figure 2A The DRAM embodiment in this example shows data rows 204 and tag rows 206. Such and other numbers and arrangements of data rows and tag rows in a DRAM, readily designed as consistent with the teachings herein, can operate as a multipath set-associative cache. In this example, each memory bank 202 of the DRAM is an array of 8 mat 208 × 64 mat 208, and each mat 208 is an array or subarray of 512 bits × 1024 bits (see [link to documentation]). Figure 6 Each mat 208 totals 512kb. In this example, each mat 208 includes redundant rows at the top and bottom, a label row adjacent to the upper redundant row or alternatively adjacent to the lower redundant row, and regular rows (i.e., data rows), using 32 rows of data per set. In various embodiments, the label row 206 may be located at the top, bottom, or elsewhere of the mat 208. Various arrangements of the label row, redundant rows, and regular data rows are possible. In some embodiments, labels in the lower half of a segment (or memory bank 202) are assigned to data in the upper half of the segment (or memory bank 202). Labels in the upper half of a segment (or memory bank 202) are assigned to data in the lower half of the segment (or memory bank 202). There are 16 memory banks 202 in each of channels A and B of DRAM0, and 16 memory banks 202 in each of channels A and B of DRAM1. The 32 tags in each of the four memory banks 202 in DRAM 0 can cache a 4×32kB set of hit data with 4×16 bytes.

[0066] Figure 2C The diagram illustrates what is suitable for Figure 2AThis is a version of a distributed tag in a DRAM embodiment. Regular word lines (depicted in solid lines) and tag word lines (depicted in dashed lines) exist in mat 210. Data on the regular word lines can be read when the tag read bit is 0. Data on the tag word lines can be read when the tag read bit is 1. In this embodiment, in a given mat 210, cached data can have multiple regular word lines, while tag data can have a single tag word line.

[0067] refer to Figure 1A , Figure 1B ,as well as Figures 2A to 2C Various embodiments have some or all of the following architectural details.

[0068] Optimizing the distribution of data units corresponding to one pathway in a multi-path cache (e.g., a 64B transaction within a 4kB pathway) can be based on a trade-off between tag-matching access speed, random data access to individual units, and access to the entire 4kB block versus allowing continuous data flow through memory. For example, Figure 1A The multipath set-associative cache DRAM implementation distributes cached data to all paths through all memory banks. Figure 2A The multipath set-associative cache DRAM implementation in the example distributes cached data to a given path through only some memory banks. Figure 1A One of the versions or its variants can have simultaneous activation for tags and data, while Figure 2A Another version or its variant can wait until a comparison has a match to activate the row for cached data access. Figure 1A One version of it is advantageous for random access to data, while Figure 2A Another version of the medium is beneficial for memory utilization and continuous streaming.

[0069] Associating path addresses with column addresses allows for complete overlap between label row activation and data row activation; however, when accessing the full path used for population (e.g., fetching and eviction), it requires accessing more memory banks or performing multiple row accesses within the same memory bank. Population involves bringing data into the cache and then finally clearing invalid data.

[0070] Tags are copied as needed, allowing tag matching to be performed locally where matching information is required. Multiple storage banks each have copies of the tags for the pathways within that storage bank that can cache data, so that any storage bank actually containing tag matching can generate the data. See, for example... Figure 2A ,Should Figure 2AThis illustrates tag hits across four banks of a multi-path set-associative cache DRAM. Each bank has tag matches for a path, and cached data for that path is read in parallel across all four banks. Each bank contains data for all paths in the set, but not complete data for each path. Figure 1A In this process, tags are copied 32 times (each of the 32 storage units has a complete set of duplicate tags), and data is accessed through the pathways of the sets in parallel, allowing all cached data of the pathway to be used at once, for a total of 32 storage units. Figure 2A In this process, the tags are copied four times (each of the four storage banks has a duplicate set of the tag group, each of the other groups in the four storage banks has another duplicate set of another tag group, and so on), and these four storage banks are accessed multiple times to obtain all the cached data for that path.

[0071] The comparison logic is located near the edge of the array. For example, the comparators and comparison blocks (see Figure 3) are positioned along the edge of the DRAM cell mat array in the memory bank. This location shortens the signal path, thereby maintaining low parasitic signal line loads, such as reducing resistance and capacitance, and may also reduce the occurrence of signal reflections.

[0072] Feedback to the controller (e.g., hit or miss) is performed using available feedback pins (alarm (ALERT_n), data mask inversion (DMI), etc.). For example, feedback to the controller can be accomplished using available pins that have a DRAM-to-controller orientation and are not needed when using DRAM in cache mode. An example is when modifying LPDDR4 DRAM to cache DRAM (see...). Figure 11 Use the DMI pin when modifying DDR4 DRAM, or the ALERT_n pin when modifying GDDR5 or GDDR6 DRAM. Another example is using the Error Detection Code (EDC) pin when modifying GDDR5 or GDDR6 DRAM.

[0073] Writing the invalid bit and writing data can begin simultaneously. The invalid bit is written along with the tag data, and the data to be cached is written to the location (or path) in the cache memory associated with the tag. If the same bank of DRAM is used for both tag data and cached data, it may take two column cycles to push the closing of the tag row out of one column cycle.

[0074] Redundancy: Tag lines can be prioritized for replacement or always doubled.

[0075] DRAM has dual uses and can be configured as cache DRAM in one mode (e.g., multipath set-associative cache mode) or as standard DRAM in another mode (e.g., standard DRAM mode). Reference Figure 7 An embodiment of the mode register is shown and described to select between a standard DRAM mode and a cached DRAM mode. The mode selection controls address multiplexing and data multiplexing, which differ between the two modes.

[0076] If a redundant row for each mat is assigned as a tag row, it is possible to change the implementation of the peripheral logic solely from the standard DRAM design layout. A standard DRAM design layout has multiple redundant rows, one of which can be designated as a tag row without altering the design layout of the mats or the row arrangement. The peripheral logic can then implement changes to address decoding and data multiplexing, as well as add comparators and comparison blocks to the N-path set associated cache operations for the tag row and the remainder of the DRAM.

[0077] In one embodiment, the most recent tag match information can be stored in DRAM to allow subsequent access to the same path without another comparison. If cached DRAM is used according to a closed-page policy, the tag and data line (path) will be closed after one access. Cached DRAM can also be used according to an open-page policy, where the data line (path) remains open after an initial cache hit. In this case, a contiguous column command can be used to access additional columns of the open line. An explicit precharge command will close the data line, and the next access will restart the tag comparison.

[0078] This allows for configurability of the tag size and the bits allocated to the address and metadata. (See reference) Figure 7 An embodiment of a mode register is shown and described for selecting a bit length for tags and comparators corresponding to the configurable number of paths in a cache associated with a multipath set.

[0079] One embodiment is a DRAM device supporting cache operations, wherein cache line tags are held in dedicated row storage elements adjacent to associated data row storage elements on individual physical word lines, these dedicated row storage elements being accessed via two associated operations. The first operation accesses the data of the tag in the tag match and accesses the data row storage element. The second operation retrieves the tag data, uses this data to point to the column of the data, and accesses the cached data. These two operations are described in reference... Figure 4A and Figure 5A The timing shown and described overlaps. Therefore, for one of the two associative operations, data and cache accesses overlap. For example, see... Figure 1A The design allows concurrent access to label rows and data rows, and label matching activates column decoding for cached data access.

[0080] For the other of two associative operations, data and cache accesses do not overlap. See, for example... Figure 2A The design in the code allows tag matching to occur before opening another row for accessing cached data.

[0081] One embodiment of the multipath set-associative cache has the following characteristics: a 2GB cache size. This is the amount of DRAM available for caching data from storage-level memory. For other embodiments, it is easy to design caches of other sizes.

[0082] Two 8Gb LPDDR4 DRAM dies. In one embodiment, this is the number of dies that can be used for a multipath set-associative cache. Other numbers of dies are readily available for other embodiments.

[0083] 32 paths. This is the number of paths or addressable locations in each set of the cache memory, where data from the storage-class memory can be cached in a multi-path (32 paths in this embodiment) set-associative cache. Other numbers of paths are readily available for other embodiments.

[0084] 4kB per channel. This is the amount of data that can be cached in each channel or line in a 32-channel set-associative cache or an N-channel set-associative cache. Other amounts of data per channel are readily available for other embodiments.

[0085] 64B transactions. This is the amount of data that can be read from or written to each cache memory in each (read or write) transaction. Other amounts of data per transaction are readily available for other embodiments.

[0086] Operation compatible with Low Power Double Data Rate Version 4 (LPDDR4) signaling and timing: One burst with a 16-bit burst length on two 16-bit channels (32B per channel). In one embodiment, standard timing and signaling for LPDDR4 DRAM are used for both standard DRAM operations and N-path set-associative cache operations when enabled. In another embodiment, the DRAM uses standard timing and signaling operations for LPDDR4 DRAM for N-path set-associative cache operations, but this is not optional for standard DRAM operations without N-path set-associative cache operations. Other embodiments may use standard timing and signaling for other types of DRAM, such as those that are easily designed.

[0087] Addressing, which includes address decoding and address line multiplexing, is easy to design for use in one embodiment for the following configuration, and in other embodiments for other variations.

[0088] 2GB cache memory is 2 31Byte = (divided into) 2 5 There are 10 pathways, each with a cache memory of 2. 12 Bytes and sets have 2 14 Therefore, the 14-bit set address identifies this set, and the size of this set is 128kB. Within the 4kB path, there are 64 = 2 6 A possible 64B transaction. In the case of a cache hit, the 6b offset address identifies the transaction.

[0089] For a 1TB SCM size, each location in the SCM has 40 bits of address space. The matching address in the tag is 40 bits – 14 bits (number of sets) – 12 bits (path size) = 14 bits. Adding two status bits makes the tag 2 bits = 16 bits. The overhead of 2 bits of tag space per 4KB of data (in this example, 32-path set associated cache per path) is very small for tag storage. Even when tags are stored in multiple locations in DRAM for parallel access, the overhead is well below 1%.

[0090] Addressing, including address decoding and address line multiplexing, is readily designed for the following configuration in one embodiment, while in other embodiments it is designed for variation. Addressing to 1b of the device, 3b of the memory bank, and 10b of row blocks belonging to a set identifies a region of 32 1kB rows in each sub-memory bank. In this embodiment, four sub-memory banks are used simultaneously.

[0091] Depending on the method and location of cached data, and design decisions regarding address multiplexing, bank selection, row decoding and row enabling, and column decoding and column enabling, various possible timing sequences exist for accessing cached data. Multiple embodiments for timing can be combined in a single device. In one embodiment, there is no overlap between tag comparison and access to cached data in DRAM. The address of the data set from main memory (e.g., SCM) that can be cached is divided into bit fields, and one field (i.e., the tag field of the address) is presented to a comparator for tag matching. The tag is read and compared with the tag field. Tag matching generates a portion of the address of the cached data in DRAM, while another portion of the address in the cached DRAM comes from the address of the data set in main memory. Access in DRAM can only be performed after all address bits are available.

[0092] In other embodiments, access to a tag for tag matching (i.e., the tag row) and access to cached data in DRAM may completely or partially overlap. Some address bits in the address set of data in main memory begin to be used for bank selection and row enable in the data rows of DRAM in parallel with or immediately after accessing the tag row. Tag matching generates a partial address of the cached data in DRAM, and this is used for column decoding to access the cached data.

[0093] This document presents various embodiments with different numbers of tag address bits for comparison, the number of paths in the set, the number of flag bits, the arrangement of address bits for column decoding and row decoding, the design and function of the sensing amplifier, etc. It should be understood that these are examples, and other embodiments can be easily designed based on these examples.

[0094] Figure 3A The illustration shows a circuit system for cache hits and addressing and accessing data in a set of paths already cached in a multipath set-associative cache in one embodiment. The tag `mat` is the same as the ordinary data `mat` in the memory bank. For example, in the illustrated embodiment, each `mat` 108 is a 512-bit × 512-bit array or subarray (see...). Figure 6 Each mat contains a total of 256 kilobits (kb). The tag array allows access to 16kb rows from memory and 256b columns from rows. Each 256b column contains 16 16b tags (106), where each tag may be associated with the accessed physical address A. P [39:27] Field matching. Using nine address bits A for row decoding. R [14:6] and the six address bits A used for column decoding 306 R [5:0] Access tags. Tags are read in parallel and compared in parallel, where each tag access provides 16 bits, of which 13 bits are presented to comparator 304 in comparison block 302 for comparison with A. P [39:27] The address field is compared, and the other three bits are flag bits (e.g., invalid, valid, parity). Other embodiments may use 14 bits for address comparison, leaving two flag bits. It is easy to design other numbers of tag bits, address comparison bits, and flag bits.

[0095] Figure 3B The diagram illustrates the use of set bits from tag data to access cached data pointed to by a matching tag in column decoding. Matching tag 106 is HIT, and set numbers (from 0 to 15) Set[3:0] are passed from comparison block 302 to the column access path of the memory, column decoding 306. In the illustrated circuitry, each comparison block 302 is hardwired with a set number N, which is enabled on the set number bits Set[3:0] by a hit in that comparison block 302. The matching data is then accessed. In various embodiments (e.g., see...), the matching data is accessed. Figure 1A , Figure 2A , Figure 9 and Figure 10 Comparison and data access can occur in parallel across multiple memory banks 102.

[0096] Figure 3CThe illustration shows an SCM and the address space in DRAM for accessing data cached in a multipath set-association cache DRAM according to one embodiment. In one embodiment, the SCM physical address is 40 bits A. P [39:0], to address SCM 2 40 1 byte (1TB). To access cached data, the 13 most significant bits (A) are used. P [39:27] Used as column address bit A for caching data C The tag field [5:2] generates Set[3:0]. Block address (15 bits of the SCM physical address) A P [26:12] Row address A used for caching data R [14:0]. The five most significant bits (A) of the sub-block address of the SCM physical address. P [11:6] Send to device address A D [1:0] provides two bits for the storage address A of the cached data. B [2:0] provides three bits. The last two bits of the sub-block address of the SCM physical address are the column address A of the cached data. C [1:0] provides two additional bits. Column decoding 306 uses address bit A. C [5:2] and A C [1:0]. After the tag comparison, all sets are available in the DRAM cache.

[0097] Figure 4A This illustration shows the timing details of normal access to data 402 and modified access to cached data 404 in one embodiment. Copies of 4kB blocks in the DRAM (Cache Memory) are distributed across 32 memory banks 102 for better performance. In the illustrated embodiment, the 4kB blocks are composed of mats, and each mat 108 is a 512-bit × 512-bit array or subarray (see [link to illustration]). Figure 6 Each mat contains a total of 256 kilobits (kb). It is easy to design other arrangements of memory volumes, blocks, and mats. Column decoding 306 is referenced above. Figure 3A The labels described above are in line 106 of the label mat file, as referenced above. Figure 1A As described. Variations of other embodiments are easily designed.

[0098] Figure 4AThe clock signal CK is shown. This clock signal CK is grouped via the command / address CA (clock), command and address (controller to DRAM), data bus DQ (data for DRAM used in normal mode), and memory bank control bank (internal activity of DRAM used in normal mode) for normal (direct) access 402 to read data. Below, these are grouped via the memory bank control bank (internal activity of DRAM used in cache mode) and the data bus DQ (data for DRAM used in cache mode) for modifying (cache) access 404 to read cached data. According to the memory bank control, for normal (direct) access 402, firstly, the memory bank is activated and sensed, then read and transferred to the interface Xface. The Xface in the timing diagram indicates the data being moved from the array inside the DRAM to appear on the external DQ (read) (e.g., ...). Figure 4A ) or from entering the external DQ to being located in the array (write) (e.g., Figure 5A The time spent.

[0099] The timing of a normal (direct) access 402 read delivers two columns of data within 41ns, where activating and sensing the memory bank 102 takes 36 clock cycles or 18ns, reading takes 8 clock cycles or 4ns, and transmitting to the interface (xface) takes 8 clock cycles or 4ns. Figure 4A and Figure 5A CA[5:0] is shown as an example of transmitting commands and addresses from the controller to the DRAM in accordance with the LPDDR4 protocol. If the DRAM of other protocols is modified to be used as cache DRAM, these functions can be accomplished using explicit address pins and separate command pins (such as RAS, CAS, and WE).

[0100] According to tag activity, for Modify (Cache) Access 404, firstly, the tag is activated and sensed, then the tag data is read and compared, where a tag hit or miss is determined. The CA input packet for Normal (Direct) Access 402 is the same as the CA input packet for Modify (Cache) Access 404. Overlapping with this activity, according to memory control, the memory is activated and sensed, where a tag hit provides the set bits Set[3:0], then the cached data is read and transmitted to the interface Xface. The timing of Modify (Cache) Read Access 404 is 51ns, where activating and sensing the tag takes the same 36 clock cycles or 18ns as reading data in Normal Access 402, followed by tag comparison taking 12 clock cycles or 6ns. The timing of cached data access overlaps, where activating and sensing memory 102 takes the same 36 clock cycles or 18 ns as activating and sensing a tag, but overlaps with it. Subsequently, reading takes the same 8 clock cycles or 4 ns, and transferring to the interface (Xface) takes the same 8 clock cycles or 4 ns. The additional 10 ns (total time for accessing and reading cached data) includes 4 ns for column access and 6 ns for 16b comparison.

[0101] Figure 4B The diagram illustrates the use of Figure 4B The read timing shown reads cached data from one bank of the multipath set-associative cache DRAM while simultaneously reading more cached data from other banks in parallel. In one embodiment, copies of 4kB blocks of data from the SCM are distributed across 32 banks in the DRAM (cache) memory (see example...). Figure 1A To achieve better performance, the 256-bit tag data is read in parallel from memory using selected address bits for mat (and tag row) selection and column decoding 306. In one embodiment, a tag match of one tag in tag 106 generates a hit indication (i.e., cache hit, tag match, or tag hit, see [link]). Figure 3A The set bits (Set[3:0]) and the invalid, valid, and parity bits are used. The set bits are combined with the specified address bits of column decoder 306, and 256 bits of cached data are read from the memory bank. Similar activity occurs across 32 memory banks of DRAM, resulting in parallel reads of 4kB blocks of cached data. For including... Figure 2A Other embodiments of the multipath set-associative DRAM cache, including the one shown, are readily available for development of variations with different bit widths and different addressing and timing schemes.

[0102] Figure 5AThis illustration shows the write timing details of normal data access 502 and modified cached data access 504 in one embodiment. Copies of 4kB blocks in the DRAM (Cache Memory) are distributed across 32 memory banks 102 for better performance. In the illustrated embodiment, the 4kB blocks are composed of mats, and each mat 108 is a 512-bit × 512-bit array or subarray (see [link to illustration]). Figure 6 Each mat has a total of 256 kilobits (kb). It is easy to design other arrangements of memory banks, blocks, and mats. Column decoding 306 is as described above. Figure 3A The labels described above are in line 106 of the label mat file, as referenced above. Figure 1A As described. Variations of other embodiments are easily designed.

[0103] Figure 5A The clock signal CK is shown. This clock signal CK is grouped via the command / address CA (clock), command and address (controller to DRAM), data bus DQ (data for DRAM used in normal mode), and the memory bank control bank (internal activity of DRAM used in normal mode) for normal (direct) access 502 to write data. Below, these are grouped via the memory bank control bank (internal activity of DRAM used in cache mode) and the data bus DQ (data for DRAM used in cache mode) for modifying (cache) access 404 to write cached data. According to the memory bank control, for normal (direct) access 502, firstly, the memory bank is activated and sensed, then transmitted to the interface Xface for writing and column access. The timing of a normal (direct) write access 502 stores two columns of data within 34ns, where activating and sensing the memory bank 102 takes 36 clock cycles or 18ns, waiting for the cycle, transmitting to the interface (Xface) takes 8 clock cycles or 4ns, writing WR takes 8 clock cycles or 4ns, and writing column COL takes 8 clock cycles or 4ns.

[0104] According to tag activity, for modified (cache) access 504, firstly, the tag is activated and sensed, then the tag data is read and compared, where a tag hit or miss is determined. The CA input packet for normal (direct) access 502 is the same as the CA input packet for modified (cache) access 504. Overlapping with this activity, according to memory bank control, the memory bank is activated and sensed, where a tag hit provides the set bits Set[3:0]; subsequently, data is transferred to the interface (Xface) and written to the cache, taking two write cycles. The Xface in the timing diagram represents the data coming out of the array inside the DRAM and appearing on the external DQ (read) (e.g., Figure 4A) or from entering the external DQ to being located in the array (write) (e.g., Figure 5A The time spent. The second WR in the "memory bank" used to modify (cache) access 504 is used for tag write WR. There is a difference between the line labeled "tag" and the line labeled "memory bank". The line "tag" shows what happens in the tag logic, that is, for WR, tags need to be read, compared, and a new tag needs to be prepared with settings such as invalid bits. The line "memory bank" shows what happens in the array. Figure 4A and Figure 5A Both read the tags, but Figure 4A and Figure 5A The line “Storage” does not show the label being read (it is below the “Activate / Sense Storage” block because the label is read while activating and sensing the data line).

[0105] The timing of the (cache) write access 504 is modified to 44ns, where activating and sensing the tag takes the same 36 clock cycles or 18ns as reading data in normal access 502, followed by a tag read taking 8 clock cycles or 4ns, and a tag comparison taking 12 clock cycles or 6ns. This is immediately followed by a write to wr taking 8 clock cycles or 4ns. The tag is read (rd) and (if hit) written back (wr), where the invalid bit is set and parity is adjusted. The hit or miss status is returned to the controller. If a miss occurs, the 64-byte write and tag write invalidation operations are canceled. The additional 10ns (total time for accessing and writing cached data) includes 4ns for column access and 6ns for 16b comparison. The 16b tag is written back, where the "invalid" flag is set, which may increase the tRC of tag access in some embodiments.

[0106] Figure 5B The diagram illustrates the use of Figure 5B The write timing shown writes cached data to one bank of the multipath-set associated cache DRAM, while more cached data is written in parallel to other banks. In one embodiment, copies of 4kB blocks of data from the SCM are distributed across 32 banks of the DRAM (cache) memory for better performance (see example...). Figure 1A The 256-bit tag data is read in parallel from the memory using the selected address bits for mat (and tag row) selection and column decoding 306. In one embodiment, a tag match of one tag in tag 106 produces a hit (i.e., cache hit, tag match, or tag hit, see [link]). Figure 3AThe instruction and set bits Set[3:0], along with invalid, valid, and parity bits, are used. The set bits are combined with the specified address bits of column decoder 306, and 256 bits of cached data are written to memory. If a hit is found, a newly prepared tag with invalid bits and adjusted parity is also written back to memory. Similar activity occurs across 32 DRAM banks, resulting in 4kB blocks of cached data being written in parallel. For including Figure 2A Other embodiments of the multipath set-associative DRAM cache, including the one shown, are readily available for development with different numbers of bits and different addressing and timing schemes.

[0107] Figure 6 The illustration shows DRAM cells 604 and sense amplifiers 608 in a mat 602, which can be used in an embodiment of DRAM. The DRAM cells 604 are arranged in rows 612 and columns 610. Each DRAM cell 604 has a MOSFET and a capacitor that can store charge. DRAM cells 604 in a given row 612 are activated by word lines 614 of that row 612 and read out onto bit lines 616 of that column 610 of the DRAM cells 604. All bit lines 616 are precharged by bit line precharge 606, and selected DRAM cells 604 are read out via sense amplifier 608. The sense amplifier 608 may have other circuitry. Typically, the sense amplifier has three functions: precharge, readout, and connection to the array data lines, and various sense amplifier designs are readily implemented for various embodiments. Notably, in this design, each mat 602 has a sense amplifier 608 dedicated to the DRAM cells 604 of mat 602. Therefore, the tag mat row 106 can be arranged with a mat set 602 and the data row 104 associated with the tag (i.e., as the tag row) and other mat sets 602, such that each tag and the tag mat row 106 is coupled to a sensing amplifier (i.e., the sensing amplifier 608 of the mat set 602 that constitutes the tag mat row 106), and the associated data row is coupled to a different sensing amplifier (i.e., the sensing amplifier 608 of those mat sets 602 that constitute the data mat row 104), and access to the tag can overlap with access to the cached data pointed to by the tag.

[0108] In various embodiments, there are different ways to avoid sense amplifier conflicts. Figure 1A and Figure 1BThere may be tag mats and data mats that are adjacent to each other, so they may need to have double sense amplifier stripes only at that boundary, and nowhere else in the memory. Figure 2 and other possible embodiments do not require doubling the normally interleaved sense amplifiers because they are constructed in a way that tags are never adjacent to their associated data. Operating adjacent mats simultaneously in a 6F2 open bitline architecture requires the overhead of a virtual array to provide supplemental bitlines, and therefore has considerable overhead.

[0109] Figure 7 The diagram illustrates a mode register 702 for selecting between a standard DRAM mode and a multipath set-associative cache mode with adjustable tag and comparator bits. One or more mode registers 702 control an address multiplexer (mux) 704, a data multiplexer (mux) 706, and a comparator 304. Address bits used for row decoding and row enable, and column decoding and column enable in tag access and cached data access, as well as for tag comparison (see [link]). Figures 3A to 5B The number and location of the data bits are selected by address multiplexer 704 and / or data multiplexer 706 according to mode selection. In various embodiments, there is a specified range of bit lengths for tags and comparator 304, which corresponds to a range of the number of paths in the multiplexed set-associated cache. Mode register 702 can select the number of bits used for tag comparison or the number of paths in the set. For example, a higher-performance solution can be converted from 32 paths to 16 paths, where the number of paths in the set is doubled. The same DRAM can be used as standard DRAM, which is compatible with the specified DRAM timing and signaling and has the comparator disabled, or as a comparator-enabled multiplexed set-associated cache, and can even switch back and forth between the two modes. In other embodiments, fuse logic can be used instead of mode register 702 to enable or select some or all of these features.

[0110] Figure 8A This is a flowchart illustrating an operational method for a multipath set-associative DRAM cache that can be practiced through the various embodiments described herein. For example, hardware, software executing on a processor, firmware, or a combination thereof can use... Figure 3A , Figure 4A or Figure 5A The timing and electrical pathways, or variations thereof, are shown to perform these various actions in order to execute... Figures 1A to 7 This illustrates access to cached data in the DRAM or its variants. In action 802, the tag lines of the DRAM are read. The tag lines are read in parallel, and these tag lines can be accessed as follows: Figure 1A and Figure 2A The described location or its variants. For modified (cached) access, a 404 error occurs. Figure 4AAn example of the read timing of action 802 is shown, where tag activity indicates activation / sensing of the tag, followed by reading (rd).

[0111] In action 804, the address bits of a set that can be cached in DRAM (e.g., from main memory or SCM) are presented to the comparator. The address bits can be selected as follows: Figure 7 or Figure 8B Control is performed as shown or its variations. For modification (caching) access, a 404 error is returned. Figure 4A An example of the read timing for action 804 is shown, where the tag activity displays a read (rd) followed by a comparison.

[0112] In action 806, the address bits of the set are compared with the data bits (i.e., tag information bits) from the tag row in DRAM. For example... Figures 3A to 5B The comparator shown performs comparisons in parallel across the DRAM memory bank. For a modify (cache) access 404, Figure 4A The example shown is a read timing sequence for action 806, where the tag activity implements the comparison, and the tag hit provides the set bit Set[3:0].

[0113] In decision action 808, it is determined whether a cache hit exists. A cache hit is determined based on the comparison result in action 806. For modification (cache) access, a 404 error occurs. Figure 4A The diagram illustrates an example of this timing sequence, where a comparison is performed to determine whether a cache hit or a cache miss occurs. If the answer is no, there is no cache hit, and the process proceeds to action 810, reporting a cache miss. If the answer is yes, there is a cache hit, and the process proceeds to action 812.

[0114] In action 812, the address bits of the multipath set associated with the cache hit are generated. Figures 3A to 5B An example of generating a set bit Set[3:0] is shown, which shows which path of the multipath set associated with the cache has the cached data set.

[0115] In action 814, data is read from the data row using the address bits based on cache hits. Figures 3A to 5B An example of column decoding for reading cached data is shown. For Modify (Cache) Access 404, Figure 4 shows an example of this timing, where the DQ bit is read onto the DQ bus after a cache hit.

[0116] Figure 8B This is a flowchart illustrating the selection of an operating mode, which can be practiced through the various embodiments described herein. In action 820, the bit length and comparison strength of the selected tag are written to the mode register to select either standard DRAM mode or multipath set-associative cache mode.

[0117] In Action 822, the address multiplexing, data multiplexing, tag bit length, and comparator bit length are controlled based on the contents of the mode register. Figure 7 An example of a mode register that controls these aspects is shown.

[0118] Figure 9 An embodiment of a multipath set associated DRAM cache is depicted. Four memory banks 902 are shown, each with a row decoder 904. Each memory bank 902 has a column decoder 906 and a secondary sense amplifier (SA) 908. Each memory bank 902 also has cache logic 914. This example shows one multiplexer 920 for every two memory banks 902, but in other embodiments, each memory bank or every number of memory banks 902 may have one multiplexer 920. Internal command and address lines from command / address logic / re-driver 916 are connected to logic (for memory bank control, decoders, and row decoders) at the edge of an array of global command and address logic and re-driver 916. Internal data lines 910 connect the secondary sense amplifier to global data logic and re-driver 918, not to cache logic 914.

[0119] Figure 10 Another embodiment of a multipath set-associative DRAM cache is depicted. Sensing amplifier 1008 is shared between neighboring mat 1004. The edge of the memory bank requires load mat 1002. Activating word line (WL) blocks evaluate word lines sharing the sensing amplifier 1008; for example, WL b block simultaneously blocks WL a and WL c.

[0120] Multipath set-associative DRAM cache (see) Figure 10 The options for the embodiment on the left include the following: Option A) mat 1 and mat n store tags, mat 1 stores tags for data in mat 2 to mat n / 2, and mat n stores tags for data in mat n / 2+1 to mat n-1. Option B) Only mat 1 stores tags, mat 2 is a shared payload mat between mat 1 and mat 3, and data is stored in mat 3 to mat n. Option C) Tags are distributed as additional rows in all tags, tags for data in mat 1 to mat n / 2 are located in mat n / 2+1 to mat n, and tags for data in mat n / 2+1 to mat n are located in mat 1 to mat n / 2.

[0121] Multipath set-associative DRAM cache (see) Figure 10The options for the embodiment on the right side include the following. Option D) matt 1006 stores a tag (i.e., tag mat), the number of word lines of matt 1006 is different from the number of word lines of mat 1 to mat n, matt 1006 needs its own load mat, where the number of word lines is the same in order to achieve load balancing (splitting load bit lines and sharing load mat may cause process problems), and the data is stored in mat 1 to mat n.

[0122] Figure 11 A DRAM 1102 according to this embodiment is depicted, along with a memory controller 1104 that can operate the DRAM 1102 as a multi-path set-associative DRAM cache. In some versions, the DRAM 1102 operates as a multi-path set-associative DRAM, while in other versions, the DRAM 1102 can be selectively operated as either standard DRAM or multi-path set-associative DRAM (see, for example, Figure 7 (See the description of mode register 702 in the documentation). The signal lines connecting the corresponding pins on DRAM 1102 and memory controller 1104 include CK, CKE, CS, CA, DQ, DQS, and DMI (see the DRAM standard below). In this embodiment, the DMI signal and pin are reused to convey cache information and are labeled DMI / CACHE_INFO.

[0123] An example DRAM standard applicable to an embodiment of DRAM 1102 is JEDEC standard number 209-4B for LPDDR4 (Low Power Double Data Rate version 4) DRAM, summarized below. Additional information is available from the published standards.

[0124] Clock input symbols CK_t_A, CK_c_A, CK_t_B, CK_c_B. Differential clock inputs, where each channel (A and B) has its own clock pair.

[0125] Clock enable input symbols CKE_A and CKE_B. Each channel (A and B) has its own clock enable signal.

[0126] The chip select input symbols are CS_A and CS_B. Each channel (A and B) has its own chip select signal.

[0127] Command / address input symbols CA[5:0]_A, CB[5:0]_B. Each channel (A and B) has its own command / address signal.

[0128] Command / address die-terminated control input symbols ODT_CA_A and ODT_CA_B. Enables or disables die-termination on the CA pin.

[0129] Data input / output (I / O) symbols: DQ[15:0]_A, DQ[15:0]_B. Bidirectional data bus. Each channel (A and B) has its own bus.

[0130] Data strobe I / O symbols: DQS[1:0]_t_A, DQS[1:0]_c_A, DQS[1:0]_t_B, DQS[1:0]_c_B. A bidirectional differential output clock signal is used to strobe data during READ or WRITE. Each channel (A and B) has its own DQS strobe.

[0131] The data mask inversion I / O symbols are DMI[1:0]_A and DMI[1:0]_B. These bidirectional signals indicate when data on the data bus is inverted or in a normal state, or provide write data mask information to the DRAM, depending on the data inversion or data mask mode register setting. Each channel (A and B) has its own DMI signal.

[0132] Calibrate the reference symbol ZQ. Calibrate the output drive strength and termination resistance.

[0133] Power supply symbols: VDDQ, VDD1, VDD2.

[0134] Grounding reference GND symbols VSS and VSSQ.

[0135] The reset input symbol is RESET_n. Active low reset signal.

[0136] Another example DRAM standard applicable to the DRAM 1102 embodiment is JEDEC standard number 79-4 for DDR4 (Dual Data Rate version 4) DRAM, summarized below. Pins or signals with similar or identical names in the LPDDR4 and DDR4 standards perform similar or identical functions. Additional information is available according to the published standards.

[0137] Clock input symbols CK_t and CK_c. Differential clock input.

[0138] Clock enable input symbol CKE (CKE1). Activates and deactivates the internal clock signal, device input buffer, and output driver.

[0139] The chip selects the input symbol CS_n (CS1_n).

[0140] Chip ID input symbols C0, C1, C2. Select each slice of the stacked components.

[0141] On-die termination input symbol ODT (ODT1). Applicable to the selected pin in the selected configuration.

[0142] The activation command input symbol is ACT_n. Define the activation command entered together with CS_n.

[0143] Command input symbols RAS_n / A16, CAS_n / A15, WE_n / A14. The multi-function pin defines the entered command.

[0144] Input data mask and data bus inverted I / O symbols DM_n / DBI_n / TDQS_t, (DMU_n / DBIU_n), (DML_n / DBIL_n). Input mask signal, or indicator of real or inverted data.

[0145] The memory group input symbols are BG0 to BG1. These define which of the following commands—valid, read, write, or precharge—is applied to the memory group.

[0146] The memory address input uses symbols BA0 to BA1. It defines which of the following commands is applied: valid, read, write, or precharge.

[0147] Address inputs are symbols A0 to A17. They provide the row address for activation commands and the column address for read / write commands.

[0148] Automatic precharge input symbol A10 / AP. Address A10 is sampled during read / write commands to determine automatic precharge.

[0149] Burst chopping input symbol A12 / BC_n. Address A12 is sampled during read / write commands to determine the burst chopping.

[0150] Active low asynchronous reset input symbol RESET_n.

[0151] Data input / output (I / O) symbol DQ. Bidirectional data bus.

[0152] Data strobe I / O symbols: DQS_t, DQS_c, DQSU_t, DQSU_c, DQSL_t, DQSL_c. Output: Read data; Input: Write data.

[0153] The terminal data strobe output symbols are TDQS_t and TDQS_c. The termination resistor function, data masking function, or data mask inversion are enabled based on the mode register.

[0154] Command and address parity input. Input symbol: PAR. Even parity is supported.

[0155] Alarm I / O symbol ALERT_n. A multi-functional alarm for CRC errors, command and address parity errors, and connectivity tests.

[0156] Connectivity test mode is enabled by inputting the symbol TEN. This enables connectivity test mode operation.

[0157] No connection symbol NC.

[0158] DQ power supply symbol VDDQ.

[0159] DQ grounding supply symbol VSSQ.

[0160] Power supply symbol VDD.

[0161] Grounding supply symbol VSS.

[0162] DRAM activation power supply symbol VPP.

[0163] The reference voltage supply symbol for CA is VREFCA.

[0164] The reference pin for ZQ calibration is supplied under the symbol ZQ.

[0165] Figure 12 A controller 1302 using DRAM 1304 with cache mode according to this embodiment is described. Solutions to potential limitations of cache systems with tagged SRAM on the controller are proposed. Figure 12 The components in the reference Figures 1A to 11 In the various embodiments described, tag capacity is added to the DRAM in a special subarray (mapping) group. In cache mode, the DRAM accesses the tags and selects the correct data (or indicates a miss). The tag and data spaces are interleaved for optimal performance.

[0166] In various embodiments, the controller 1302 may be a CPU, a separate system-on-a-chip (SoC) separate from or on a module of the CPU and memory, as long as the controller 1302 is located in the communication path of the SCM. The controller can be connected to DRAM, and the buses connecting the controller to the SCM include DDRX, SATA, PCIe, etc. The system can be connected to a single DRAM 1304 with one channel, a single DRAM 1304 with two channels, or two DRAMs 1304 each with one channel, rather than... Figure 12 The example shown illustrates two DRAMs, each with two channels, working together. Furthermore, buffers can be present in the path of the direct-connect bus to expand capacity.

[0167] In one embodiment described below as a configuration example XA, copies of the 4kB group 1202 in the SCM (physical) memory stack 1312 are stored in DRAM (cache) memory distributed across 32 banks in the DRAM for optimal performance. DRAM 1304 is shown as two DRAM components (which may be ICs, multi-chip IC packages, logic cells, etc.) each having two channels 1306 coupled to controller 1302. Other configurations described below may also be available. Figure 12 The system shown is implemented as follows. Controller 1302 has a read queue 1310 and address field manipulation 1308, which manipulates address bits in various formats or schemes depending on the configuration. Some embodiments have a single configuration, while others have multiple configurations. In some embodiments, the physical address field is placed in a CA packet for DRAM. A control register field selects the cache configuration used. Address field manipulation 1308 uses logic, multiplexing or other circuitry, or software or firmware executed on the processor, or combinations thereof that are easily designed in accordance with the teachings herein.

[0168] When the system is in cache mode, the cache read logic of controller 1302 sends a cache read command across the memory bus to DRAM 1304 (see [link]). Figures 15A to 17 The cache result logic receives a response to cached data from DRAM 1304 via SCM 1312 or generates a miss indication by comparing a tag comparison field with a tag field stored in DRAM 1304. In the event of a miss, the cache fetch logic sends a cache fetch command 1312 (shown as the SCM stack) across another memory bus to the SCM. The cache fetch command causes SCM 1312 to send data associated with the tag field to controller 1302. In response to the cache fetch command, the cache fetch logic writes the data it received from SCM 1312 into DRAM 1304 as cached data. In some embodiments, the cache read logic retryes the cache read command to DRAM 1304.

[0169] The table below compares system configurations of DRAM with cached mode. For the various configurations XA, YA, ZA, ZB, "A" = Overlapping tag access / data access, and "B" = Non-overlapping tag access / data access. Overlapping means that data access and tag access overlap; in other words, data access begins before tag access is completed. Non-overlapping means that data access and tag access do not overlap; in other words, data access begins after tag access is completed. Non-overlapping tag access / data access takes longer than overlapping tag access / data access. "Z" = Data "path" held within a single memory cell, "Y" = Data "path" held within multiple memory cells of a single device, and "X" = Data "path" held within multiple memory cells of multiple devices. Figure 13A The document describes a configuration for XA where data paths are maintained across multiple storage banks on multiple devices and tag access / data access overlaps. Figure 13B The document describes a configuration YA in which data paths are maintained across multiple storage units on multiple devices and tag access / data access overlaps. Figure 13C The document describes a ZA configuration where data channels are kept in a single storage unit and tag access / data access overlaps. Figure 13D Configuration ZB is described, where the data path is maintained in a single storage unit and tag access / data access is non-overlapping. Tags are maintained in associated tag storage units, whose XYZ distribution options are the same as the data in the database. That is, in configuration X, the data path and associated tag storage units are maintained in multiple storage units across multiple devices. In configuration Y, the data path and associated tag storage units are maintained in multiple storage units across a single device. In configuration Z, the data path and associated tag storage units are maintained in a single storage unit.

[0170]

[0171]

[0172]

[0173] The above comparisons and the following details are for a specific embodiment, but specifications such as access time may differ in any given implementation. Traditional (uncached) access time is approximately 36ns. Configuration assumptions include two channels per DRAM, 8GB per DRAM, 32 bytes per column, 64 columns per row (2kB rows), 16K rows per memory bank, and 8 memory banks per channel. RC Approximately 50 ns, t FAWThe time to access the cache is approximately 40ns, with a DQ bit rate of 4Gb per second, 16 paths (sets) per DRAM cache line, and 4kB per DRAM cache line (set). The hit access time for all overlapping tag access / data access configurations XA, YA, and ZA is approximately 46ns, which is shorter than the 64ns hit access time for non-overlapping tag access / data access configuration ZB. However, configuration ZB has the lowest per-bit cost energy for non-overlapping tag access / data access after a miss, and the same lowest tag overhead as configuration ZA, at 0.05%. Other characteristics and trade-offs of the various configurations are readily apparent in the table.

[0174] Figure 13A Example XA of a system configuration in which data paths are maintained across multiple memory banks 102 across multiple devices and tag access / data access overlaps is depicted. In this example, 4kB groups 1202 are copied from SCM (physical) memory and distributed across 32 memory banks 102. The 16-bit tags of the distributed 4kB groups 1202 are copied in 32 tag memories 1502, one tag memory per memory bank 102. The 16-bit tags of the other paths of the 4kB groups (at the same cache group address) are maintained in the same column of a row in memory bank 102. The transport blocks of the other paths of the 4kB groups (at the same cache group address) are maintained in the same row as the 64-byte transport blocks in two 32-byte columns of the row maintained in memory bank 102. Configuration XA can benefit DRAM cache systems using wide data buses across multiple channels.

[0175] Figure 13B Example system configuration YA is depicted where data paths are maintained within multiple memory banks 102 of a single device and tag access / data access overlaps. In this example, 4kB groups 1202 are copied from SCM (physical) memory and distributed across four rows in each of the eight memory banks 102 of a channel (channel D). In this example, the 16-bit tags of the distributed 4kB groups 1202 are copied in eight tag memories 1502 (one tag memory in each of the eight memory banks of a channel (channel D)). The 16-bit tags of the other paths of the 4kB groups (at the same cache set address) are maintained in the same column in one row of memory bank 102. The transport blocks of the other paths of the 4kB groups 1202 (at the same cache set address) are maintained in the same four rows of memory bank 102 as the 64-byte transport blocks maintained in two 32-byte columns in each of the four rows. Configuration YA may be advantageous for multiple sequential data accesses at consecutive addresses in a single channel. Compared to other configurations, the average power consumption of multiple sequential data accesses at consecutive addresses in a single channel may be lower.

[0176] Figure 13C Example system configuration ZA is depicted where data paths are maintained within a single memory bank 102 and tag access / data access overlaps. In this example, 4kB groups 1202 are copied from SCM (physical) memory and distributed across 32 rows of a single memory bank 102 across one channel. 16-bit tags 106 for 64-byte transport blocks reside in associated tag memory 1502 within memory bank 102. Tags in other memory banks 102 are not duplicated. 16-bit tags for the other paths of the 4kB groups (located at the same cache set address) are maintained in the same column of a row in tag memory 1502 within memory bank 102. Transport blocks for the other paths of the 4kB groups 1202 (located at the same cache set address) are maintained in the same 32 rows as 64-byte transport blocks maintained in two 32-byte columns of each of the 32 rows in memory bank 102. Configuration ZA eliminates the need for duplicate tags, thus consuming fewer rows of tag memory. The configuration of X, Y, and Z corresponds to different schemes used for addressing rows, columns, memory banks, and channels in DRAM, and may affect access time and power consumption.

[0177] Figure 13D Example system configuration ZB is depicted where data paths are kept within a single memory bank 102 and tag access / data access does not overlap. In this example, 4kB groups 1202 are copied from SCM (physical) memory and distributed across 32 rows of a single memory bank 102 across one channel. 16-bit tags 106 for 64-byte transport blocks reside in associated tag memories 1502 within memory bank 102. Tags in other memory banks 102 are not duplicated. 16-bit tags for other paths of the 4kB groups (at the same cache set address) are kept in the same column of a row in tag memories 1502 within memory bank 102. Transport blocks for other paths of the 4kB groups 1202 (at the same cache set address) are kept in different rows within memory bank 102. 64-byte transport blocks are kept in two 32-byte columns across two rows within memory bank 102. Configuration ZB eliminates the need for tag duplication, thus reducing row consumption in tag memory. Configuration B has a longer access time than configuration A, but may consume less power.

[0178] Figure 14A Describes the use of, for example Figure 13A The system configuration example XA shown is Figure 12The address field in controller 1302 is manipulated by 1308. For 2GB DRAM, the DRAM byte address 1604 is derived from the controller (physical) address 1602 for a 1TB memory (e.g., an SCM, whose individual groups 1202 are cached in DRAM) by comparing the address field control 1308 on controller 1302 with the tag on the DRAM. The tag field A from controller (physical) address 1602... P [39:27], DRAM generates a set bit Set[3:0] through tag comparison. This set bit Set[3:0] is used for the four address bits A of DRAM byte address 1604. C [5:2]. Group address A of controller (physical) address 1602. P All 15 bits of [26:12] are used for row address bit A of DRAM byte address 1604. R [14:0]. Controller 1302 derives the channel address bit A of DRAM byte address 1604 from the upper two and lower three bits of the subgroup address of controller (physical) address 1602. D [1:0] and memory address A B [2:0]. The last two bits of the subgroup address of controller (physical) address 1602 provide the remaining two column address bits A of DRAM byte address 1608. C [1:0]. Furthermore, the bottom five bits of the subgroup address of controller (physical) address 1602 provide five bits for the sub-column address used for 64-byte read / write to complete DRAM byte address 1604.

[0179] Figure 14B Describes the use of, for example Figure 13B The system configuration example YA shown Figure 12 The address field in controller 1302 controls 1308. For 2GB DRAM, the DRAM byte address 1606 is derived from the controller (physical) address 1602 for a 1TB memory (e.g., an SCM, whose individual groups 1202 are cached in DRAM) by comparing the address field control 1308 on controller 1302 with the tag on the DRAM. The tag field A from controller (physical) address 1602... P [39:27], DRAM generates a set bit Set[3:0] through tag comparison. This set bit Set[3:0] is used for the four address bits A of DRAM byte address 1606. C [5:2]. From controller (physical) address 1602, group address A P The upper two bits of [26:25] provide channel address bit A for controller 1302. D [1:0]. From controller (physical) address 1602, group address A PThe controller 1302 provides the upper 13 bits of the row address A for the DRAM byte address 1606, representing the lower 13 bits or the remaining 13 bits of [24:12]. R [14:2]. Subgroup address A of controller (physical) address 1602. P The top three bits of [11:9] are manipulated to be the memory address bit A of DRAM byte address 1606. B [2:0]. Subgroup address A of controller (physical) address 1602. P The last two digits of [8:7] provide the row address A of DRAM byte address 1606. R The bottom two bits of [1:0]. Subgroup address A of controller (physical) address 1602. P The last two digits of [6:5] are used as column address A in DRAM byte address 1606. C The bottom two bits of [1:0]. And the bottom five bits of the subgroup address of controller (physical) address 1602 provide five bits for the sub-column address for 64-byte read / write to complete DRAM byte address 1606.

[0180] Figure 14C Describes the use of, for example Figure 13C The system configuration example ZA shown is Figure 12 The address field in controller 1302 manipulates 1308. Figure 14C The solid arrows in the diagram depict the tag comparison process on the DRAM. The tag field, which is part of the physical address, is compared with tags stored in the DRAM, and if a match is found, one of 16 sets (the Set[3:0] field) is selected—the Set field forms part of the DRAM address used to select data. For a 2GB DRAM, the DRAM byte address 1608 is derived from the controller (physical) address 1602 of a 1TB memory (e.g., an SCM, whose individual groups 1202 are cached in DRAM) via the address field on controller 1302. The tag comparison on the DRAM is derived from the tag field A on controller (physical) address 1602. P [39:27], DRAM generates a set bit Set[3:0] through tag comparison. This set bit Set[3:0] is used for the four address bits A of DRAM byte address 1608. C [5:2]. That is, the set bits from the tag comparison are used for the four column address bits to retrieve cached data. From group address A of controller (physical) address 1602. P [26:12], Controller 1302 exports channel address bit A of DRAM byte address 1608. D [1:0], Memory address bit A B [2:0] and the previous 10 row address bits AR [14:5]. The other five row address bits A of DRAM byte address 1608 R [4:0] Subgroup address A from controller (physical) address 1602 P The top five bits of [11:0] are derived. The middle two bits of the subgroup address of controller (physical) address 1602 provide the other two column address bits A of DRAM byte address 1608. C [1:0]. The next five bits of the subgroup address of controller (physical) address 1602 provide five bits of the subcolumn address of DRAM byte address 1608.

[0181] Figure 14D Describes the use of, for example Figure 13D The system configuration example shown is ZB. Figure 12 The address field in controller 1302 controls 1308. For 2GB DRAM, the DRAM byte address 1610 is derived from the controller (physical) address 1602 of a 1TB memory (e.g., an SCM, whose individual groups 1202 are cached in DRAM) by comparing the address field control 1308 on controller 1302 with the tag on the DRAM. The tag field A from controller (physical) address 1602... P [39:27], DRAM generates a set bit Set[3:0] through tag comparison. This set bit Set[3:0] is used for the four rows above address A of DRAM byte address 1608. R [14:11]. From controller (physical) address 1602, group address A P [26:12], Controller 1302 exports channel address bit A of DRAM byte address 1608. D [1:0], Memory address bit A B [2:0] and the last 10 row address bits A R [10:1]. The lowest address bit A of DRAM byte address 1608. R [10:0] Subgroup address A from controller (physical) address 1602 P

[11] The topmost bit. The last 6 bits of the subgroup address of controller (physical) address 1602 provide the six column address bits A of DRAM byte address 1608. C [5:0]. The lower five bits of the subgroup address of controller (physical) address 1602 provide five bits of the subcolumn address of DRAM byte address 1608.

[0182] Figure 15A Describing as Figure 13D The system configuration example shown is the command format for ZB. (Figure 15 and...) Figure 15BRedundancy occurs because there are two column accesses per row access; that is, other DRAM components or subsystems might set this ratio to one column access per row access, and there would be no redundant fields in the command sequence. The ACT command is followed by RD command 0 and RD command 1. In this example, the ACT command indicates that it has the least significant tag field 1702 bits A. P [30:27]. The first read command (RD command 0) has the most valid tag field 1704 bits A. P [39:31]. Subsequent RD commands in the same line within the same group do not require a tag field, for example, RD command 1. This is because the read command for each protocol, which has two parts, already specifies the tag bits required to complete the tag field when combined with the tag field 1702 from the ACT command in RD command 0, thus the tag field in RD command 1 is redundant.

[0183] Figure 15B Depicting Figure 13C , Figure 13B and Figure 13A The system configuration example shown illustrates the command format for ZA, YA, and XA. The ACT command is followed by RD command 0 and RD command 1. In this example, the ACT command only contains the line address. The first read command (RD command 0) has the most significant tag field 1706 bits A. P [39:31] and the least valid tag field 1708 bits A P [30:27]. Subsequent RD commands on the same line within the same group (e.g., RD ​​command 1) do not require a tag field. This is because the read command for each protocol, which has two parts, has already specified all the tag bits required to complete the tag field in RD command 0, so the tag field in RD command 1 is redundant.

[0184] Figure 15C Describing as Figure 13D , Figure 13C , Figure 13B and Figure 13A The system configuration examples shown are the command formats for ZB, ZA, YA, and XA. The ACT command is followed by RD command 0 and RD command 1. In this example, the first read command (RD command 0) indicates that it has the most valid tag field 1710 bits. P [39:33]. The next read command (RD command 1) has the least significant tag field 1712 bits A. P [32:27]. If only 32 bytes of access are needed, the same column can be read twice. Figure 15A and Figure 15B The command format examples differ; RD command 1 does indeed require providing the remaining tag bits to complete the tag field, therefore there is no redundant tag field. (See reference) Figures 15A to 15CThe concept of a "shared" tag field can refer to a tag field (from a physical address) that spans (…). Figure 15A In this context, row commands and column commands are separate, or specified by a single column command, and later by (…). Figure 15B The second column of commands is used, or across ( Figure 15C The two consecutive columns of commands are separated, meaning that any of these alternatives allows for a minimum number of CA pins.

[0185] Figure 16 The timing details of a read operation with cache misses in a DRAM with cache mode are depicted. The controller queue receives the next read (RD) transaction (T) 1720 at (1). This is shown as the read address on the address bus (ADR). An ACT / RD access to the DRAM is performed at (2) 1722. The access is cancelled at (3) by a MISS tag 1724. This is signaled via a status (ST) signal from the DRAM to the controller. The transaction (T) status is retry, which the system accomplishes by sending the status to the execution unit via the ST2 signal at (4). The transaction (T) moves to the retry buffer at (5) until the fetch operation has returned to read (RD) data. At (6), the system initiates a concurrent fetch procedure to obtain a group with the missed read (RD) data from external memory (XMEM). The concurrent fetch procedure is shown as fetching a new group 1726. At (7), the memory controller selects the group to be removed from the cache memory (to make room for the new group) and initiates a concurrent eviction process. This is shown as eviction of old group 1728. At (8), the system continues to process transactions concurrently. At (9), the fetch access to the 4kB fetch buffer is completed, and the 4kB buffer has been moved to DRAM. At (10), the fetch is signaled to the memory controller in the background field of the status packet to indicate that the fetch is complete and the original read can be retried.

[0186] Figure 17 This describes the timing details of 64-byte reads, normal (direct) access 1802, and modified (cached) access 1804 for DRAM in cached mode. At a high level (and referring to the table at

[00125] ), the differences between the four configurations affect hit access latency, tag overhead cost, and bandwidth and energy / bits for accessing a 4KB group after a miss (evict / fetch). These interact in complex ways, but they cannot be optimized simultaneously, which is why four configurations are needed to illustrate the optimal point for each metric. For modified (cached) access 1804 and normal (direct) access 1802, the CA input packets are in the same position. The ACT command, followed by RD command 0 and RD command 1, follows the sequence and has the above reference. Figures 15A to 15CThe bits described depend on the system configuration. Configure XA, YA, and ZA (see the table for a comparison of system configuration examples XA, YA, ZA, and ZB). Figures 13A to 13C , Figures 14A to 14C and Figure 15B Tag access and data access overlap, where Figure 17 The timing for configuration XYZ-A is shown as modified (cache access) 1804. Configuration ZB (see the table comparing system configuration examples XA, YA, ZA, and ZB) Figure 13D , Figure 14D and Figure 15A Tag access and data access do not overlap, where Figure 17 The timings for configuration ZB are shown as modified (cache) access 1804. This timing can be compared to a faster data read in normal (direct) access 1802, which is used to directly read DRAM without cache access.

[0187] refer to Figure 16 A to Figure 16 C and Figure 17 In one embodiment, there are three commands: ACT followed by two consecutive column commands, used to combine two 32-byte read or write data accesses into a 64-byte access. The two consecutive column commands are a label column access operation and a subsequent data column access operation. Both column accesses use the same label. That is, the label comparison field of the label column access operation is also used for the label comparison field of the data column access operation.

[0188] Figure 18 This is a flowchart depicting the controller reordering process of a DRAM cache experiencing cache misses. (Forward Reference) Figure 19 Various system components participate Figure 18 The action described in [1] begins with action 1902, [1] obtaining the next transaction “T”, which can be a read, for example, a 64-byte read (see [1]). Figure 19 (on the left), or write, for example, 64-byte write (see left). Figure 18 (Right side).

[0189] For a 64-byte read, the process proceeds to action 1904, [2r] reading DRAM 1304 and determining whether there is a hit or a miss transaction T. If there is a hit, the process proceeds to action 1906 and returns the read data. If there is a miss [3], the process proceeds to action 1908.

[0190] For a 64-byte write, the process proceeds to action 1930, [2w] writing to DRAM and determining whether there is a hit or a miss transaction T. If a hit occurs, the process proceeds to action 1932, writing 64 bytes of data to DRAM 1304 (based on a cache hit), and then proceeds to action 1934, setting an invalidation flag for a 128-byte subgroup. If a miss occurs [3], the process proceeds to action 1908.

[0191] For action 1908[8], controller 1302 performs the action of marking the state of transaction T as “retry”[4],[5] holding transaction T,[6] initiating a concurrent “acquisition” process,[7] initiating a concurrent “eviction” process, and[8] continuing concurrent processing of the transaction.

[0192] For [6] retrieval, the process proceeds to action 1910, [6a] which begins accessing group B of 4kB in the SCM. T (Including transaction T target) and transmitted to the 4kB acquisition buffer 2006 in controller 1302. The process flow branches in parallel, where the extraction is completed, proceeding to action 1922 [7c], and continuing to action 1912 [6b], transferring the 4kB group B T The data is transferred from the 4kB buffer in controller 1302 to DRAM 1304. Next, action 1914 [6c] returns transaction T to the retry queue in read retry buffer 2022, and action 1916 [6d] terminates the fetch process. In some embodiments, once the fetch from SCM to DRAM is complete, memory controller 1302 must retrieve the data from DRAM 1304. The controller may wait for acknowledgment, or in some versions rely on a predetermined time interval, and then request data from DRAM 1304 again via the process of action 1902. In some embodiments, controller 1302 reads the data as it moves from SCM to the DRAM cache, thus preventing a separate DRAM read from being required once the data is in the DRAM cache.

[0193] For [7] eviction, the process proceeds to action 1918, [7a] selecting 4kB group B in the DRAM cache. E Evicting (Group B) E Group B T The same A exists in 16 sets at this address. P [26:12] Address). Next, the process proceeds to action 1920 [7b] which will transfer 4kB group B. E (Including the transaction) along with 32 invalid flags (one invalid flag per 128 bytes of data) are transferred from DRAM to the 4kB evict buffer 2004 in controller 1302. The process proceeds to action 1922 [7c] waiting for access B. TAnd transmit from SCM to control via (optional) acquisition process. The process proceeds to action 1924 [7d] to expel 4kB from controller 1302 from buffer 2004, group B of 4kB. E (Including transactions) are transmitted to the SCM. Optionally, 32 invalid flags can control the writing of every 128 bytes of data. In action 1926, the [7e] eviction process is terminated.

[0194] The following describes the ejection selection options used in various embodiments.

[0195] [7a.1] The eviction selection can be random.

[0196] [7a.2] Several byte subgroups from the 32 128-byte subgroups of the 16 target 4kB groups can be sampled. The number of high invalid flags can be used to indicate how much of each 4kB group has been used. The group with the highest usage can be selected. If SCM endurance is an issue, the group with zero high invalid flags can be selected.

[0197] [7a.3] Similar to the above, but a touch icon can be used to indicate how many 4kB blocks have been used.

[0198] Figure 19 It describes the execution Figure 18 The diagram illustrates the system action of a controller using DRAM with cached mode. In the DRAM system, DRAM 1304 is shown as DRAM 0 and DRAM 1 in the first DRAM arrangement, with more DRAM in the second DRAM arrangement. Each DRAM 1304 has two channels: channel A and channel B in DRAM 0, and channel C and channel D in DRAM 1. The SCM memory stack 1312 has 4kB groups 1202 in the SCM (physical) memory. Inside the controller 1302, transaction formatting and manipulation 2010 is coupled to DRAM 1304 via interface 2012 (represented as xface). Status / control 2014 is coupled to the transaction formatting and manipulation 2010 for the interface 2012 coupled to DRAM 1304. Transaction formatting and manipulation 2010 is coupled to write queue 2016, read queue 2018, write retry buffer 2020, and read retry buffer 2022. Another transaction formatting and manipulation 2002 is coupled to the SCM memory stack 1312 via another interface 2024. Another state 2008 is coupled to transaction formatting and manipulation 2002. Transaction formatting and manipulation 2002 is coupled to eviction buffer 2004 and fetch buffer 2006, which fetch buffer 2006 is also coupled to transaction formatting and manipulation 2014 of DRAM 1304.

[0199] Regarding the above reference Figure 18 The described cache miss was achieved by using... Figure 18 and Figure 19 The corresponding numbers in parentheses in the diagram easily reveal the controller order of the DRAM cache.

[0200] Figure 20 The touch flag enhancement for the eviction decision process is described. A touch flag is added to a 16-bit tag flag word 2106, which then has 12 bits for tag comparison and four flag bits for invalid, valid, parity, and touch 2104. When a 4kB block is loaded from the SCM via the fetch process, the touch 2104 (and invalid) bit is initially low. A 64-byte read transaction sets the touch 2104 flag in the local tag flag word 2106 high (and adjusts the parity) for the accessed 128-byte sub-block (similar to when the invalid flag is set by a 64-byte write transaction). The touch bit, which is cleared when the 4kB block is first written to the DRAM cache (i.e., loaded from the SCM), is set when the 4kB block or a portion is read from the DRAM cache to notify the system that the cached 4kB block is being read and is therefore worth retaining. A 4kB block that has been written to the DRAM cache so that the touch bit has been cleared but still shows the touch bit as cleared later is likely an eviction candidate because it has not been read and is therefore not worth keeping. The invalid bit set by the write transaction notifies the system that cached data has been written and should therefore be written back to the SCM. This is useful in write-back to the cache. Both the invalid bit and the touch bit serve a purpose in the eviction decision process. As mentioned above, the touch bit can be used to identify eviction candidates, and the invalid bit can be used to ensure that cached data that has been written (i.e., overwritten) at that position in the cache is considered an eviction candidate before being written back to the SCM.

[0201] Figure 21A A 4kB cache line distributed across memory banks / channels / devices according to this embodiment is depicted. In this example, the data and associated tags of each 4kB cache line are distributed across all available memory banks 102 (e.g., 32 memory banks from two DRAMs and two channels from each DRAM, with eight memory banks 102 per channel). This allows the 4kB cache line 2202 to be transferred between the DRAM and the controller as quickly as possible. This makes the fetch and eviction processes easier to schedule. Figure 22A illustrates how the 4kB cache line 2202 in the physical SCM address space is distributed across memory banks 102 of multiple channels and multiple devices to minimize transport time during the fetch and eviction processes.

[0202] Figure 21BThis illustration depicts 4kB cache line tags and data for 16 sets (paths) located in the same DRAM row according to this embodiment. In this example, the data blocks transferred by a path (e.g., two 64-byte transfer blocks) are located in the same row. A path is multiple sets of data with the same low-to-mid address but different high addresses. The tags for the paths are located in a single column (e.g., 16×16 bits). This generates tags after row / column access and tag comparison, and data after two more column accesses. Row access for tags overlaps with row access for data. Figure 21B This demonstrates how to access 16 aliased 4kB cache lines in the physical SCM address space to find cache lines with matching addresses 39:27. In this example, these 16 4kB cache lines 2202 occupy address A. P The 16 sets at [26:12]. A P [26:12] Same row address but A P [39:27] The matching addresses are different. It should be noted that each row of the tag matching contains 64 columns of 16×16-bit tags associated with 64 rows in data storage 102.

[0203] The methods, systems, and devices described above can be implemented or stored in a computer system. The methods described above can also be stored on a non-transitory computer-readable medium. The devices, circuits, and systems described herein can be implemented using computer-aided design tools available in the art and are embodied in computer-readable documents containing software descriptions of such circuits. This includes, but is not limited to, one or more elements and components of DRAM. These software descriptions may be descriptions of behavior, register transfers, logic components, transistors, and layout geometry. Furthermore, the software descriptions can be stored on a storage medium or transmitted via a carrier wave.

[0204] Data formats that can be used to implement such descriptions include, but are not limited to, formats supporting behavioral languages ​​such as C, formats supporting register-transfer level (RTL) languages ​​such as Verilog and VHDL, formats supporting geometric description languages ​​(such as GDSII, GDSIII, GDSIV, CIF, and MEBES), and other suitable formats and languages. Furthermore, data transmission of such files on machine-readable media can be accomplished electronically via various media on the Internet or, for example, via email. It should be noted that physical files can be implemented on machine-readable media such as 4mm magnetic tape, 8mm magnetic tape, 3-1 / 2 inch floppy disk media, CDs, DVDs, hard disk drives, solid-state drives (SSDs), etc.

[0205] The foregoing description of the illustrated embodiments of this disclosure, including the content described in the abstract, is not intended to be exhaustive or to limit this disclosure to the precise forms disclosed. While specific embodiments and examples of this disclosure have been described herein for illustrative purposes, those skilled in the art will recognize that various equivalent modifications are possible within the scope of this disclosure. Other embodiments may have a different order, additional layers, or fewer layers than the illustrated embodiments.

[0206] Various operations are described sequentially as a plurality of discrete operations in a manner most conducive to understanding this disclosure; however, the order of description should not be construed as implying that these operations necessarily depend on the order. Specifically, these operations need not be performed in the order of presentation.

[0207] As used herein, the terms “above,” “over,” “below,” “between,” and “on” refer to the relative position of a material layer or component with respect to other layers or components. For example, a layer deposited above, above, or below another layer may be in direct contact with the other layer or may have one or more intermediate layers. Furthermore, a layer deposited between two layers may be in direct contact with both layers or may have one or more intermediate layers. In contrast, the first layer “on” a second layer is in direct contact with that second layer. Similarly, unless otherwise explicitly stated, a feature deposited between two features may be in direct contact with the adjacent feature or may have one or more intermediate layers.

[0208] The terms “example” or “exemplary” are used herein to mean as an example, instance, or illustration. Any aspect or design described herein as “example” or “exemplary” is not necessarily to be construed as superior to or advantageous to other aspects or designs. Rather, the use of the words “example” or “exemplary” is intended to present concepts in a specific manner. As used in this application, the term “or” is intended to mean inclusive “or” rather than exclusive “or.” That is, unless otherwise stated or clear from the context, “X includes A or B” is intended to mean any natural inclusion arrangement. That is, if X includes A, X includes B, or X includes both A and B, then “X includes A or B” is satisfied under any of the foregoing examples. Additionally, unless otherwise stated or clear from the context to the singular form, the articles “a” and “an” used in this application and the appended claims can generally be construed as meaning “one or more”. Furthermore, unless so described, the consistent use of the terms “an embodiment” or “one embodiment” or “one example” or “one embodiment” is not intended to refer to the same embodiment or example. The terms “first,” “second,” “third,” “fourth,” etc., used in this article are intended as labels to distinguish different elements, and their numerical designation may not necessarily have an ordering meaning.

Claims

1. A memory controller, comprising: The cache read logic sends a cache read command to the first memory component across the first memory bus. The cache read command includes: a command opcode, the row address of the row storing the tag field associated with the data, and a tag comparison field. The cache result logic receives a response from the first memory unit across the first memory bus, the response having the data, or having a miss indication resulting from comparing the tag comparison field with the tag field stored in the row of the first memory unit; and The cache retrieval logic, in response to the miss indication, sends a cache retrieval command across the second memory bus to a second memory unit, the cache retrieval command causing the second memory unit to send the data associated with the tag field stored in the row of the first memory unit to the memory controller.

2. The memory controller of claim 1, wherein the cache retrieval logic writes the data received from the second memory unit in response to the cache retrieval command into the first memory unit.

3. The memory controller according to claim 1, wherein the cache read logic retryes the cache read command of the first memory component.

4. The memory controller according to claim 1, wherein: The first memory component includes dynamic random access memory (DRAM), which can be configured as a cache memory and has the row storing the tag field; and The second memory component includes a storage-class memory (SCM).

5. The memory controller according to claim 1, wherein the cache read command includes: The first command includes the first label field and the line address field; The second command includes the second label field and the column address field; as well as The third command includes redundant label fields.

6. The memory controller according to claim 1, wherein the cache read command includes: The first command includes the line address field; The second command includes two label fields; as well as The third command includes redundant label fields.

7. The memory controller according to claim 1, wherein the cache read command includes: The first command includes the line address field; The second command includes the first label field and the column address field; as well as The third command includes the second label field.

Images