Multi-level system memory with different cache structures and a memory controller supporting concurrent lookups of the different cache structures

Abstract

An apparatus is described. The apparatus includes a memory controller connected through an interface to a multi-level system memory having first and second different cache structures. The memory controller has circuitry to service a read request by concurrently performing a lookup in the first and second different cache structures for a cache line targeted by the read request.

Description

[0001] Priority claim

[0002] This application claims priority under 35 USC § 365(c) to U.S. Application No. 15 / 394,550, filed on December 29, 2016, entitled “MULTI LEVEL SYSTEM MEMORY HAVING DIFFERENT CACHING STRUCTURES AND MEMORY CONTROLLER THATSUPPORTS CONCURRENT LOOK-UP INTO THE DIFFERENT CACHING STRUCTURES,” which is hereby incorporated by reference in its entirety. Technical Field

[0003] The field of the invention relates generally to computing science; and more particularly, to multi-level system memory having different cache structures and a memory controller that supports concurrent lookups to the different cache structures. Background Art

[0004] Computing system designers are increasingly interested in improving the efficiency of the systems they design. System memory, also known as main memory, is increasingly an area where efficiency improvements can be made. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] A better understanding of the present invention may be obtained from the following detailed description in conjunction with the accompanying drawings, in which:

[0006] Figure 1 A computing system having multiple levels of system memory is shown;

[0007] Figure 2 shows a multi-level system memory with an associative and direct-mapped cache structure;

[0008] Figure 3 A main memory controller designed to access a near memory associative direct mapped cache structure and far memory is shown;

[0009] Figure 4 Shown by Figure 3 A method performed by a memory controller;

[0010] Figure 5 A computing system is shown. DETAILED DESCRIPTION

[0011] 1.0 Multi-level system memory

[0012] One of the ways to improve system memory performance is to have multiple levels of system memory. Figure 1 An embodiment of a computing system 100 is shown having a multi-tiered or multi-level system memory 112. According to various embodiments, a smaller, faster near memory 113 may be utilized as a cache for a larger far memory 114.

[0013] The use of cache memory for computing systems is well known. In the case of using near memory 113 as a cache, near memory 113 is used to store additional copies of those data items in far memory 114 that are expected to be called more frequently by the computing system. The near memory cache 113 has a shorter access time than the lower-level far memory 114 areas. By storing the more frequently called items in the near memory 113, the multi-level system memory 112 will be observed to be faster because the computing system will often read the items stored in the faster near memory 113. For implementations using write-back techniques, the copies of data items in near memory 113 may contain data that has been updated by the central processing unit (CPU) and are therefore more recent than the data in far memory 114. The process of writing 'dirty' cache entries back to far memory 114 ensures that such changes are not lost.

[0014] According to some embodiments, for example, near memory 113 exhibits reduced access time by having a faster clock speed than far memory 114. Here, near memory 113 may be a faster (e.g., shorter access time) volatile system memory technology (e.g., high performance dynamic random access memory (DRAM)) and / or static random access memory (SRAM) memory cells co-located with memory controller 116. In contrast, far memory 114 may be a volatile memory technology implemented with a slower clock speed (e.g., a DRAM component that receives a slower clock) or, for example, may be a non-volatile memory technology that is slower (e.g., longer access time) than volatile / DRAM memory or any technology used for near memory.

[0015] For example, the far memory 114 may include emerging non-volatile random access memory technologies such as (to name a few possibilities) phase change based memory, three-dimensional cross-point memory devices or other byte-addressable non-volatile memory devices, "write-in-place" non-volatile main memory devices, memory devices using chalcogenides, single-level or multi-level flash memory, multi-threshold level flash memory, ferroelectric based memory (e.g., FRAM), magnetic based memory (e.g., MRAM), spin transfer torque based memory (e.g., STT-RAM), resistance based memory (e.g., ReRAM), memristor based memory, universal memory, Ge2Sb2Te5 memory, programmable metallization cell memory, amorphous cell memory, Ovshinsky memory, etc.

[0016] Such emerging non-volatile random access memory technologies typically have some combination of: 1) higher storage density than DRAM (e.g., by being constructed in a three-dimensional (3D) circuit structure (e.g., a cross-point 3D circuit structure); 2) lower power density than DRAM (e.g., because they do not require refresh); and / or 3) slower access latency than DRAM but still faster than traditional non-volatile memory technologies such as FLASH. The latter characteristic specifically allows various emerging byte-addressable non-volatile memory technologies to be used in the role of main system memory rather than the role of traditional mass storage devices (the traditional architectural location of non-volatile storage devices).

[0017] Regardless of whether far memory 114 is comprised of volatile or non-volatile memory technology, in various embodiments, far memory 114 acts as real system memory in that it supports finer granularity data access (e.g., cache line) rather than larger based access associated with conventional non-volatile mass storage devices (e.g., solid state drives (SSDs), hard disk drives (HDDs)), and / or far memory 114 otherwise acts as (e.g., byte) addressable memory out of which program code executed by the CPU's processor(s) operates. However, far memory 114 may be inefficient when accessing far memory 114 for a small number of contiguous bytes of data (e.g., less than 128 bytes), an effect that may be mitigated by the presence of near memory 113 operating as a cache capable of efficiently handling such requests.

[0018] Because near memory 113 acts as a cache, near memory 113 may not have a formal addressing space. Instead, in some cases, far memory 114 defines a separate addressable memory space of the main memory of the computing system. In various embodiments, near memory 113 acts as a cache for far memory 114, rather than acting as a last-level CPU cache. In general, CPU caches are optimized for servicing CPU transactions, and will add significant costs (such as cache snooping overhead and cache eviction flow in the event of a hit) to other memory users such as devices with direct memory access (DMA) capabilities in a peripheral control hub (PCH). In contrast, memory-side caches are designed to handle accesses to system memory, regardless of whether they come from the CPU, from a peripheral control hub, or from some other device such as a display controller.

[0019] In various embodiments, the memory controller 116 and / or the near memory 113 may include local cache information (hereinafter referred to as "metadata") so that the memory controller 116 can determine whether a cache hit or a cache miss occurs in the near memory 113 for any incoming memory request. The metadata may also be stored in the near memory 113.

[0020] In the case of an incoming write request, if there is a cache hit, the memory controller 116 directly overwrites the data associated with the request (e.g., a 64-byte CPU cache line) over the cached version in the near memory 113. Likewise, in the case of a cache miss, in an embodiment, the memory controller 116 also writes the data associated with the request to the near memory 113, potentially first fetching any missed portion of the data from the far memory 114 that is required to constitute the minimum data size that can be marked in the metadata as valid in the near memory 113, a technique known as 'underfill'. However, if an entry / entries of writable content in the near memory cache 113 have been allocated to a different system memory address / addresses, and the entry / entries contain data that is more recent than the data retained in the far memory 114 (i.e., it is dirty), then the data occupying that entry is evicted from the near memory 113 and written to the far memory 114.

[0021] In the case of an incoming read request, if there is a cache hit, the memory controller 116 responds to the request by reading a version of the cache line from the near memory 113 and providing it to the requester. In contrast, if there is a cache miss in the near memory 113, the memory controller 116 reads the requested cache line from the far memory 114 and, in some embodiments, not only provides the cache line to the requester, but also writes another copy of the cache line to the near memory 113. In many cases, the amount of data requested from the far memory 114 and the amount of data written to the near memory 113 will be greater than the amount of data requested by the incoming read request. Using larger data sizes from the far memory or to the near memory increases the probability of a cache hit to a nearby memory location for subsequent transactions.

[0022] In general, cache lines may be written to and / or read from near memory and / or far memory at different levels of granularity (e.g., writes and / or reads may occur at only cache line granularity (and, e.g., byte addressability where writes and / or reads are handled internally within a memory controller), byte granularity (e.g., true byte addressability where a memory controller only writes and / or reads one or more bytes identified within a cache line), or granularities in between). Additionally, note that the size of cache lines maintained within near memory and / or far memory may be larger than the size of cache lines maintained by a CPU-level cache. Different types of near memory cache architectures are possible (e.g., direct mapped, set associative, etc.).

[0023] The physical implementation of near memory and far memory in any specific system may vary with the embodiment. For example, a DRAM near memory device may be coupled to a first memory channel, while an emerging non-volatile memory device may be coupled to another memory channel. In yet other embodiments, the near memory and far memory devices may communicate with the host-side memory controller via the same memory channel (in even further embodiments, the near memory and far memory devices are deployed on the same DIMM card). The near memory and / or far memory devices may be integrated in the same (one or more) semiconductor chip packages as the processing core and the memory controller, or may be integrated outside of (one or more) semiconductor chip packages.

[0024] The far memory devices may be directly coupled to the far memory controller, where an independent communication channel couples the far memory controller to the host-side memory controller (e.g., a multi-drop memory channel bus, a point-to-point link, etc.). The far memory controller performs various tasks, such as are specific to emerging non-volatile memory devices used for far memory. For example, the far memory controller may apply signals to far memory devices with special voltage and / or timing requirements, may manage the movement / rotation of more frequently accessed data to less frequently accessed memory cells (transparent to the system memory addressing organization of the system from the perspective of the processing core under a process known as wear leveling), and / or may identify groups of bad memory cells and prevent them from future use (also known as bad block management).

[0025] The host-side memory controller may be able to communicate with multiple far memory controllers and corresponding far memory devices as memory expansion "plug-ins". In various embodiments, the memory expansion plug-in solution may be implemented using point-to-point links (e.g., one PCIe link per plug-in). Non-extended far memory (provided as part of the basic original system) may or may not be implemented using point-to-point links (e.g., a dual in-line memory module (DIMM) card with near memory devices, far memory devices, or a combination of near and far memory devices may be plugged into a double data rate (DDR) memory channel originating from the main memory controller).

[0026] 2.0 Multi-level system memory with associative and direct-mapped cache

[0027] Figure 2 A specific multi-level system memory implementation 200 is shown in which two different cache schemes are supported using near memory 213. The first cache scheme is a faster tag lookup cache and is implemented as an associative cache 221. The second cache scheme is a slower tag lookup cache and is implemented as a direct mapped cache 222.

[0028] Here, as understood in the art, an associative cache is capable of mapping any particular system memory address to different storage device slots of multiple caches (in the case of an associative cache, the multiple slots for any particular address are referred to as "ways" for that address). As such, a particular cache line may be stored in any slot to which its system memory address is mapped (in various embodiments, the slot is identified by performing a hash function on the system memory address). In contrast, in the case of a direct-mapped cache, a particular system memory address is mapped to only one slot in the cache.

[0029] Therefore, an associative cache is more efficient in being able to hold the most frequently accessed cache lines. That is, in the case of an associative cache, if a highly utilized cache line is mapped to a slot that is already consumed by another highly utilized cache line, then it may be able to be stored in one of the other slots to which the address of the cache line in the associative cache is mapped (which may also hold, for example, less frequently used cache lines). In contrast, in the case of a direct-mapped cache, if a highly utilized cache line is mapped to a slot that already holds another highly utilized cache line, then one of the highly utilized cache lines will not be held in the cache.

[0030] However, the improved cache efficiency of the associative cache is achieved at the expense of significantly more complex memory controller circuitry 223 required to implement the associative cache. Figure 2 As observed in , such circuitry 223 includes a tag lookup table 225. Tag lookup table 225 includes an entry for each slot in the associated cache 221. Each entry in lookup table 225 holds a tag for a specific cache line currently held in the corresponding slot of the entry in the associated cache 221. The tag is a unique identifier for a specific cache line.

[0031] In response to receiving a read or write request, the associated cache logic 223 will perform a lookup of the entry in table 225 corresponding to the slot to which the address of the cache line that is the subject of the read or write request is mapped in the associated cache 221. If there is a "hit" (one of the tags in table 225 matches the tag of the requested address), then in the case of a read, the cache line is read from the matching slot in the associated cache 221. In the case of a write, the cache line attached to the write request is written to the matching slot.

[0032] If there is a "miss", then in the case of a read, the read request is serviced elsewhere (the requested cache line is not in the associated cache 221). In the case of a write miss, the associated cache logic 223 may choose to evict the cache line currently residing in one of the slots to which the address of the write request is mapped (e.g., according to a least recently used (LRU) algorithm) so that the cache line attached to the write request can be written to the associated cache (if so, the tag of the cache line being written will be entered into the corresponding entry in table 225).

[0033] Because table 225 includes an entry for each slot in the associated cache 221, it can become increasingly large and expensive (in terms of silicon chip surface area consumption) as the size of the associated cache 221 increases. Figure 2 In an implementation of the memory subsystem of FIG. 1 , the associative cache is made smaller than the direct-mapped cache 222 (e.g., one-third, one-fifth, one-tenth, one-fifteenth). By keeping the associative cache 221 small, the cost of the lookup table 225 and its supporting circuitry is kept to a minimum.

[0034] In contrast, the logic circuit 224 used to implement the direct-mapped cache 222 need not include a lookup table. Instead, the tag of the specific cache line stored in the direct-mapped cache slot can be stored in the slot along with the cache line. That is, unlike the associated cache 222 that stores the tag in the memory controller 216, in contrast, the direct-mapped cache can store the tag in the cache 222 itself (thus, the data width for the direct-mapped cache should be wide enough to store not only the cache line, but also the additional metadata including the tag).

[0035] In the case of a read or write request to the direct mapped cache 222, the slot to which the requested address is mapped is read from the direct mapped cache 222, and the tag included with the read from the slot is compared with the tag of the requested address. If the tags match, there is a cache hit. In the case of a read, the read cache line read from the slot is provided as a response to the read request. In the case of a write, the cache line included in the request is written to the slot. If the tags do not match, there is a cache miss, and the request is serviced elsewhere.

[0036] The memory controller 216 also includes tracking logic 227 that manages the eviction / fill of cache lines from / to the associative cache 221, the direct-mapped cache 222, and the far memory 214. Here, in various embodiments, when a cache line is evicted from the associative cache 221, it is inserted into the direct-mapped cache 222 (in an alternative embodiment, the cache line evicted from the associative cache is sent to the far memory 214). Additionally, when a cache line is evicted from the direct-mapped cache 222, it is written to the far memory 214.

[0037] In the case of a system memory read request received by the main memory controller 216, the associated cache 221 is first browsed. If there is a hit in the associated cache 216, then the cache line in the associated cache is provided as a response to the request. If there is a miss in the associated cache 221, then the direct mapped cache 222 is browsed. If the cache line sought is in the direct mapped cache 222, then the cache line read from the direct mapped cache 222 is provided as a response to the read request.

[0038] Additionally, in various approaches, a cache line provided as a response to a read request is forwarded to the associated cache logic 223 for storage in the associated cache 221. Writing the cache line to the associated cache 221 may cause the eviction of another cache line from the associated cache 221. The cache line evicted from the associated cache 221 may then be forwarded to the direct-mapped cache logic 224 for insertion into the direct-mapped cache 222. Inserting the cache line into the direct-mapped cache 222 may cause the eviction of another cache line from the direct-mapped cache 222, which is forwarded to the far memory logic 226 for writing to the far memory 214. In yet other embodiments, a read response provided from a hit in the direct-mapped cache 222 will not be forwarded to the associated cache 221. Instead, the cache line remains in the direct-mapped cache without being changed to the associated cache. In yet other embodiments, a dynamic decision is made as to which of caches 221, 222 a cache line from far memory will be entered into. That is, a cache line provided from far memory to service a read request may be entered into the associative cache 221 or into the direct mapped cache 222, depending on the results of a cache selection algorithm.

[0039] If there is a miss in the direct mapped cache 222, the read request is forwarded to the far memory logic 226. The requested cache line is fetched from the far memory 214 and provided in response to the request. Additionally, the cache line provided from the far memory 214 may be forwarded to the associated cache logic 223 for storage in the associated cache 221 (which may again cause the eviction of the first cache line from the associated cache 221, which was to be inserted into the direct mapped cache 222, which in turn causes the eviction of the second cache line from the direct mapped cache 222, which was to be inserted into the far memory 214). Alternatively (or even in combination), the cache line provided from the far memory 214 is input to the direct mapped cache 222 instead of the associated cache 221 (which may cause the eviction of another cache line from the direct mapped cache).

[0040] In the case of a system memory write request, the associated cache 221 is first scanned. If there is a hit in the associated cache 221, the cache line appended to the request is written to the same slot where the corresponding cache line already resides in the associated cache 221. If there is a miss in the associated cache, the cache line is written to the associated cache 221, which may cause another cache line to be evicted to the direct-mapped cache 222. If so, a lookup is performed by the direct-mapped cache logic 224 on the cache line that was just evicted from the associated cache 221. If the lookup results in a direct-mapped cache hit, the evicted cache line is overwritten on the corresponding cache line content in the direct-mapped cache 222. If the lookup to the direct mapped cache is a miss, then the evicted cache line is written to the direct mapped cache 222 , which causes another cache line to be evicted from the direct mapped cache 222 , which is then written to the far memory 214 .

[0041] Here, the tracking logic 227 "keeps track" of the movement of various cache lines between the caches 221, 222 and the far memory 214. Specifically, the tracking logic 227 maintains an understanding of which cache lines are inserted into the caches 221, 222, and maintains an understanding of which cache lines are evicted from the caches 221, 222. Thus, the tracking logic 227 can identify which cache lines are in a transient state to or from either cache 221, 222. In various embodiments, the tracking logic 227 can track cache lines waiting to be written to the far memory (e.g., queued in the far memory write queue 228).

[0042] Recall that direct mapped cache 222 is made larger than associative cache 221 because it does not have the cost of a memory controller side lookup to table 225. However, because associative cache 221 can perform tag lookups on-die using table 225, associative cache 221 can be considered a faster tag lookup cache than direct mapped cache 222. Therefore, as just described above, read requests are handled by first browsing the associative cache 221. If the browse to the associative cache 221 results in a miss, the direct mapped cache 222 is browsed. If the browse to the direct mapped cache 222 results in a miss, the far memory 214 is browsed. (For both read and write requests, the state of tracking logic 227 is also checked to see if the cache line being sought is in a transient state. If so, the response process takes appropriate action).

[0043] However, for those requests that result in a direct-mapped cache hit or are serviced from far memory 214, it is wasteful to first browse the associated cache 221. That is, initially browsing the associated cache 221 increases the latency of memory requests that are not serviced from the associated cache 221. In addition, because the direct-mapped cache 222 is larger than the associated cache 216, the probability of a hit in the direct-mapped cache 222 is not insignificant. Therefore, the latency of browsing the associated cache 214 (which can be a significant number of read requests that are actually serviced from the direct-mapped cache 222) degrades the observed performance of the near memory 213 cache system.

[0044] Figure 3 A more efficient system memory read request handling process is illustrated in which associative cache accesses and direct mapped cache accesses are performed within the same time frame (eg, in parallel).

[0045] Here, as in Figure 3 As observed in , when a read request is received by the memory controller 1, the memory controller proceeds to take three concurrent actions 2 in response: 1) the associated cache logic 323 performs a lookup on the associated cache (the associated cache logic looks for a tag match in the lookup table 325); 2) the direct-mapped cache logic 324 performs a lookup on the direct-mapped cache (the direct-mapped cache is read and the tag returned from the read is compared to the tag of the requested address); and 3) the tracking logic 327 checks to see if the requested cache line is in a transient state. Here, the concurrency of 1) and 2) above corresponds to concurrent lookups of the associated cache 321 and the direct-mapped cache 322, which should reduce the observed latency of the direct-mapped cache performance as just discussed above.

[0046] The action taken by the memory controller decision logic 329 then depends on the result 3 of the query 2 made to the associated cache, direct mapped cache and tracking logic blocks 323 , 324 , 327 .

[0047] Here, if the associative cache logic 323 indicates a hit in the associative cache, then the requested cache line is serviced from the associative cache and the cache line concurrently read from the direct mapped cache by the direct mapped cache logic 324 to perform the tag comparison is discarded.

[0048] In an embodiment, if a cache line is in a transient state to / from an associated cache, then it is not possible to indicate a hit for the lookup table 325 because the lookup table 325 is on the same die as the tracking logic 327. That is, the tracking logic 327 and the lookup table 325 are tightly synchronized so that a true hit or miss can be reported from the associated cache logic 323. For example, if the desired cache line is in the process of being input into or evicted from the associated cache, then the tracking logic can provide the requested cache line as a read response and the associated cache logic 323 can report a hit. If the tracking logic 327 does not recognize that the cache line is in transit and displays a miss to the lookup table 325, then the miss reported by the associated cache logic 323 can be understood to be a true miss.

[0049] Here, if the associated cache logic 323 indicates a miss in the associated cache, the direct mapped cache logic 324 indicates a hit in the direct mapped cache, and the tracking logic 327 does not indicate that the requested cache line is in a transient state, then the request is serviced from the direct mapped cache (using the cache line that was read from the direct mapped cache to perform the tag comparison operation to service the read request).

[0050] Here, for reasons discussed more fully below, because the tag information for a direct mapped cache is maintained in the actual cache rather than in the memory controller 316 logic circuitry, the tight synchronization that exists between the tracking logic 327 and the lookup table 325 (which exists with respect to the associated cache tag lookup) does not exist with respect to the direct mapped cache tag comparison. As such, there may be some form of propagation delay between the state of the tag information present in the direct mapped cache and the accuracy of that information. Therefore, a true hit from the direct mapped cache is only confirmed when the direct mapped cache logic 324 indicates a hit and the tracking logic 327 does not indicate that the cache line is in a transient state.

[0051] Still further, if the associative cache logic 323 indicates a miss in the associative cache, the direct mapped cache logic 324 indicates a miss in the direct mapped cache, and the tracking logic 327 does not indicate that the requested cache line is in a transient state, then the request is serviced by reading the requested cache line from far memory (via far memory logic 326).

[0052] As can be summarized from the above, the decision logic 329 accepts input from the tracking logic 327 to confirm the results of the tag comparison performed by the direct mapped cache logic 324. As mentioned above, because the tag information for a cache line stored in a direct mapped cache is stored in the cache itself along with its corresponding cache line, the direct mapped cache cannot keep its tag information updated as quickly as the associated cache (because the associated cache has a memory controller side lookup table 325).

[0053] Consider the case of a cache line that is cached in a direct-mapped cache but for which a decision has just been made to evict the cache line from the direct-mapped cache (e.g., because the most recent access to the cache line's slot resulted in a miss). From the time the cache line is considered evicted (which may occur once the last access to the cache line resulted in a miss) until the time the new replacement cache line and its tag are completely overwritten over the evicted cache line in the direct-mapped cache slot, the tag information contained in the slot may be technically inaccurate.

[0054] That is, there will be tag information for the cache line being evicted, and not the tag information for the replacement cache line, in the slot in the direct-mapped cache. The time frame over which such inaccurate tag information may be retained in a direct-mapped cache generally increases with increasing cache line size, since it takes longer to fully evict a larger cache line and fully replace it with a replacement cache line of equal size. Additionally, the direct-mapped cache logic 324 may prioritize servicing reads over writes.

[0055] If a read is performed by the direct-mapped cache logic 324 of a slot in servicing a read request, and the slot contains inaccurate tag information, the read result may be a "false hit." That is, the old tag information that will be overwritten matches the tag information of the read request. Meanwhile, the most recently updated copy of the evicted cache line is elsewhere in the system and may even have been updated by the new write information (e.g., located in the far memory write queue 328 or in far memory).

[0056] Thus, in an embodiment, if the direct mapped cache logic 327 indicates a direct mapped cache hit, but the tracking logic 327 indicates that the requested cache line is in a transient state, then the request is not serviced by the content just read by the direct mapped cache logic 324. In this case, the correct requested cache line is serviced from some other source.

[0057] For example, if the tracking logic 327 indicates that the requested cache line is in the process of being evicted from the direct mapped cache, then the requested cache line is found along the data path from the direct mapped cache logic 324 all the way to the far memory access logic 326 (e.g., within the far memory inbound write queue 328). In an embodiment, once a cache line has been completely overwritten by its replacement cache line in the direct mapped cache and committed to far memory, the tracking logic 327 "clears" its tracking of the cache line that has been evicted from the direct mapped cache. After this, the cache line is no longer technically in transit, and false hits are no longer possible because the tag of the evicted cache line is no longer in the direct mapped cache.

[0058] In the opposite case where the direct-mapped cache logic 324 reports a miss in the direct-mapped cache, and the tracking logic 327 indicates that the requested cache line is in the process of being inserted into the direct-mapped cache, the requested cache line may be provided from a data path that feeds into the direct-mapped cache (such as a data path from an associative cache to the direct-mapped cache). Thus, in an embodiment, if the decision logic 329 receives a miss from the associative cache logic 323, receives a miss from the direct-mapped cache logic 324, but the tracking logic 327 indicates that the cache line is about to be inserted into the direct-mapped cache, the decision logic 329 chooses to obtain the requested read line from such a data path rather than from far memory.

[0059] In an embodiment, once the correctly requested cache line is retrieved, its cache coherence state information is checked (e.g., by state checking logic, for ease of illustration, in Figure 3 314). For example, in various embodiments, a cache line has associated metadata that indicates whether the cache line is in a modified, exclusive, shared, or invalid state consistent with the MESI cache coherence protocol. For example, if the cache line is in the modified state, the main memory controller 316 may choose to write a cached copy back to the far memory 314. Here, consistent with the purpose of the MESI cache coherence protocol, in the previous discussion in which an evicted cache line is "written back" to a lower cache or far memory, such a write-back operation is performed only for cache lines in the modified state.

[0060] In a further embodiment, Figure 3The memory controller 316 is designed to handle "super lines" rather than normal cache lines. Here, just as an example, a CPU processing core may be designed to locally cache 64 bytes of data as a single cache line. However, when, for example, a cache line is input into a last-level cache that is architecturally located just above the memory controller 316 (an eviction from the last-level cache causes the evicted line to be sent to the memory controller 316), such a 64-byte cache line is packed into a larger super line, which includes 4, 8, 16, or 64 64-byte cache lines (thus, a super line is 256 bytes, 512 bytes, 1024 bytes, or 4096 bytes). In this way, in contrast to the smaller traditional cache lines, the above description of Figure 2 and Figure 3 The cache line in question is a superline.

[0061] Here, operation is as discussed above, except that the superline itself may include metadata that marks the superline as a whole as being in one of the MESI states. In addition, the metadata may further include the corresponding MESI state of each of the corresponding constituent cache lines that have been packaged into the superline. Therefore, the state check that occurs once the correct cache line has been retrieved includes a state check not only on the superline MESI state, but also on the corresponding MESI states of its respective traditional cache lines. Note that in the case of a superline, the aforementioned cache coherence problem (false hits in a direct-mapped cache because incorrect tag information remains resident in the direct-mapped cache) is more likely to occur without the algorithm discussed just above.

[0062] In a further embodiment, the decision logic 329 sends an “acknowledge” or “not acknowledge” message to the read request handling logic of the memory controller (in Figure 3 ) and / or direct mapped cache logic 324, which indicates whether a read result from a direct mapped cache read operation should be used to service the read request.

[0063] Specifically, a "not confirmed" message indicating that read information from the direct-mapped cache is not used is sent if the following conditions are met: 1) the associated cache logic 323 indicates a hit (a cache line served from the associated cache); 2) both the associated cache logic 323 and the direct-mapped cache logic 324 indicate a miss, and the tracking logic 327 indicates that the cache line is not in transit (a cache line served from a far memory); or 3) the tracking logic indicates that the requested cache line is evicted from or inserted into the direct-mapped cache (a cache line served from a data path to / from the direct-mapped cache or a destination of a cache line in transit).

[0064] In contrast, if the associated cache logic indicates a miss, the direct mapped cache indicates a hit, and the tracking logic does not indicate that the cache line is in transit, then an "acknowledge" message is sent indicating that the read information from the direct mapped cache is used to service the read request.

[0065] In a further embodiment, the read request logic of the memory controller assigns a unique ID to each incoming read request, and Figure 3 One or more of the queries 2 and corresponding responses 3 are appended with a unique ID, so that the decision logic 329 can understand not only which responses 3 correspond to which read requests in transit, but also which decision result is applied to which read request in transit by appending the unique ID to the confirmation / unconfirmation message. With the confirmation / unconfirmation message appended with the unique ID, the read handling request logic (and / or the direct-mapped cache logic) of the memory controller will know which of its multiple read requests in transit are serviced from the direct-mapped cache, and which read requests are not serviced from the direct-mapped cache.

[0066] In some embodiments, decision logic 329 resides outside of the memory controller and is positioned between the memory controller and the requesting entity (eg, between the memory controller and a processing core).

[0067] Figure 4 The method described above is shown. The method includes receiving 401 a read request for a cache line stored in a multi-level system memory having first and second different cache structures. The method also includes servicing 402 the read request by concurrently performing tag lookups for the cache line on both the first and second different cache structures.

[0068] Figure 5A depiction of an exemplary computing system 500 is shown, which may be a larger computing system such as a personal computing system (e.g., a desktop or laptop), a mobile or handheld computing system such as a tablet device or smart phone, or a server computing system. Figure 5 As observed in , the basic computing system may include a central processing unit 501 (which may include, for example, multiple general-purpose processing cores and a main memory controller deployed on an application processor or a multi-core processor), a system memory 502, a display 503 (e.g., a touch screen, a tablet), a local wired point-to-point link (e.g., USB) interface 504, various network I / O functions 505 (such as an Ethernet interface and / or a cellular modem subsystem), a wireless local area network (e.g., WiFi) interface 506, a wireless point-to-point link (e.g., Bluetooth) interface 507 and a global positioning system interface 508, various sensors 509_1 to 509_N (e.g., one or more of a gyroscope, an accelerometer, a magnetometer, a temperature sensor, a pressure sensor, a humidity sensor, etc.), a camera 510, a battery 511, a power management control unit 512, a speaker and a microphone 513, and an audio encoder / decoder 514.

[0069] The application processor or multi-core processor 550 may include one or more general processing cores 515, one or more graphics processing units 516, memory management functions 517 (e.g., memory controllers), and I / O control functions 518 within its CPU 501. The general processing cores 515 typically execute the operating system and application software of the computing system. The graphics processing unit 516 typically performs graphics-intensive functions, such as generating graphics information presented on the display 503. The memory control function 517 is connected to the system memory 502 through an interface. The system memory 502 may be a multi-level system memory with different cache structures in the faster level of system memory. The memory function 517 may be designed to perform concurrent lookups in different cache structures as described above.

[0070] Each of the touch screen display 503, communication interfaces 504-507, GPS interface 508, sensor 509, camera 510, and speaker / microphone codecs 513, 514 can all be considered as various forms of I / O (input and / or output) relative to the overall computing system, which also includes integrated peripherals (e.g., camera 510) where appropriate. Depending on the implementation, various components of these I / O components may be integrated on the application processor / multi-core processor 550, or may be located off-die or external to the package of the application processor / multi-core processor 550. The mass storage device of the computing system can be implemented using a non-volatile storage device 520 that can be coupled to an I / O controller 518 (which can also be called a peripheral control hub).

[0071] Embodiments of the present invention may include the various processes set forth above. The processes may be implemented in machine executable instructions. Instructions may be used to cause a general or special purpose processor to perform certain processes. Alternatively, these processes may be performed by specific hardware logic containing hard-wired logic for performing these processes, or by any combination of software or instruction programmed computer components or custom hardware components (such as application specific integrated circuits (ASICs), programmable logic devices (PLDs), programmable logic arrays (PLAs), or field programmable gate arrays (FPGAs)).

[0072] Elements of the present invention may also be provided as machine-readable media for storing machine-executable instructions. Machine-readable media may include, but are not limited to, floppy disks, optical disks, CD-ROMs and magneto-optical disks, flash memories, ROMs, RAMs, EPROMs, EEPROMs, magnetic or optical cards, propagation media, or other types of media / machine-readable media suitable for storing electronic instructions. For example, the present invention may be downloaded as a computer program that may be delivered from a remote computer (e.g., a server) to a requesting computer (e.g., a client) via a communication link (e.g., a modem or a network connection) by means of a data signal embodied in a carrier or other propagation media.

[0073] A device has been described above that includes a memory controller interfaced with a multi-level system memory having first and second different cache structures. The memory controller has circuitry for servicing read requests by concurrently performing lookups in the first and second different cache structures for cache lines targeted by the read requests.

[0074] In at least one embodiment of the device, the first cache structure is an associative cache and the second cache structure is a direct-mapped cache. In a further embodiment, tag information for the associative cache is stored on the memory controller and tag information for the direct-mapped cache is stored in the direct-mapped cache.

[0075] In at least one embodiment of the apparatus, tag information for the first cache is stored on the memory controller and tag information for the second cache is stored in the second cache.

[0076] In at least one embodiment of the apparatus, the memory controller includes tracking logic circuitry that identifies a cache line that is about to be inserted into or evicted from at least one of the first or second cache structures. The circuitry receives information from the tracking logic circuitry. In a further embodiment, the information is used by the circuitry to identify that a tag match for a cache line in one of the caches is invalid if the information indicates that the cache line is evicted from the one of the caches. In a further embodiment, a cache stores its tag information within a cache.

[0077] In at least one embodiment of the apparatus, the circuitry is designed to provide a cache line if the cache line has: a) a matching tag found in one of the caches; b) no matching tag in the other of the caches.

[0078] A computing system has been described that includes the apparatus described immediately above.

[0079] A method is described that includes receiving a read request for a cache line held in a multi-level system memory having first and second different cache structures. The method also includes servicing the read request by concurrently performing tag lookups for the cache line on the first and second different cache structures.

[0080] In at least one embodiment of the method, the first cache structure is an associative cache and the second cache structure is a direct mapped cache. In a further embodiment, tag information for the associative cache is stored on a memory controller and tag information for the direct mapped cache is stored in the direct mapped cache. In a further embodiment, the method further includes identifying that a tag match for a cache line in the direct mapped cache is invalid because tracking logic indicated that the cache line was evicted from the direct mapped cache.

[0081] In the foregoing description, the present invention has been described with reference to specific exemplary embodiments thereof. However, it will be apparent that various modifications and changes may be made thereto without departing from the broader spirit and scope of the present invention as set forth in the appended claims. Accordingly, the description and drawings are to be regarded in an illustrative rather than a restrictive sense.

Claims

1. A device comprising: A memory controller connected via an interface to a multi-level system memory having first and second different cache structures, the memory controller having circuitry for servicing read requests by concurrently performing lookups in the first and second different cache structures for cache lines targeted by the read requests, wherein the first and second different cache structures are memory-side caches, wherein the memory controller includes tracking logic circuitry for identifying cache lines that will be inserted into at least one of the first or second cache structures.

2. The apparatus of claim 1, wherein the first cache structure is an associative cache and the second cache structure is a direct mapped cache.

3. The apparatus of claim 2, wherein tag information for the associated cache is stored on the memory controller and tag information for the direct mapped cache is stored in the direct mapped cache.

4. The apparatus of claim 1, wherein tag information for the first cache structure is stored on the memory controller and tag information for the second cache structure is stored in the second cache.

5. The apparatus of claim 1, wherein the tracking logic circuit identifies a cache line that is about to be evicted from the at least one of the first or second cache structures, the circuit being operable to receive information from the tracking logic circuit.

6. The apparatus of claim 5, wherein the information is for use by the circuitry to identify that if the information indicates that the cache line is evicted from one of the caches, then a tag match for the cache line in the one cache is invalid.

7. The apparatus of claim 6, wherein said one cache stores its tag information within said one cache.

8. The apparatus of claim 1, wherein the circuit is configured to provide the cache line if the cache line satisfies the following conditions: a) finding a matching tag in one of the caches; b) No matching tag is found in any other cache in the cache.

9. A computing system comprising: one or more processing cores; Multi-level system memory; a memory controller interfaced to the multi-level system memory having first and second different cache structures, the memory controller having circuitry for servicing read requests by concurrently performing lookups in the first and second different cache structures for cache lines targeted by the read requests, wherein the first and second different cache structures are memory-side caches, wherein the memory controller includes tracking logic circuitry for identifying cache lines that are about to be inserted into at least one of the first or second cache structures; and Network interface.

10. The computing system of claim 9, wherein the first cache structure is an associative cache and the second cache structure is a direct mapped cache.

11. The computing system of claim 10, wherein tag information for the associated cache is stored on the memory controller and tag information for the direct mapped cache is stored in the direct mapped cache.

12. The computing system of claim 9, wherein tag information for the first cache structure is stored on the memory controller and tag information for the second cache structure is stored in the second cache.

13. The computing system of claim 9, wherein the tracking logic circuit identifies a cache line that is about to be evicted from the at least one of the first or second cache structures, the circuit being operable to receive information from the tracking logic circuit.

14. The computing system of claim 13, wherein the information is for use by the circuitry to identify that if the information indicates that the cache line is evicted from one of the caches, then a tag match for the cache line in the one cache is invalid.

15. The computing system of claim 14, wherein the one cache stores its tag information within the one cache.

16. The computing system of claim 9, wherein the circuit is configured to provide the cache line if the cache line satisfies the following conditions: a) finding a matching tag in one of the caches; b) No matching tag is found in any other cache in the cache.

17. A method performed by a memory controller, comprising: receiving a read request for a cache line stored in a multi-level system memory having first and second different cache structures, wherein the first and second different cache structures are memory-side caches; as well as The read request is serviced by concurrently performing a tag lookup for the cache line on both the first and second different cache structures and identifying a cache line to be inserted into at least one of the first or second cache structures.

18. The method of claim 17, wherein the first cache structure is an associative cache and the second cache structure is a direct mapped cache.

19. The method of claim 18, wherein tag information for the associated cache is stored on the memory controller and tag information for the direct mapped cache is stored in the direct mapped cache.

20. The method of claim 18, further comprising identifying that a tag match for the cache line in the direct mapped cache is invalid because tracking logic indicated that the cache line was evicted from the direct mapped cache.

21. A device for a memory controller, comprising: means for receiving a read request for a cache line stored in a multi-level system memory having first and second different cache structures, wherein the first and second different cache structures are memory-side caches; as well as means for servicing the read request by concurrently performing tag lookups on both the first and second different cache structures for the cache line; The device further comprises means for identifying a cache line to be inserted into at least one of the first or second cache structures.

22. The apparatus of claim 21, wherein the first cache structure is an associative cache and the second cache structure is a direct mapped cache.

23. The apparatus of claim 22, wherein tag information for the associated cache is stored on the memory controller and tag information for the direct mapped cache is stored in the direct mapped cache.

24. The apparatus of claim 22, further comprising means for identifying that a tag match for the cache line in the direct mapped cache is invalid because tracking logic indicated that the cache line was evicted from the direct mapped cache.

25. A computer readable medium having stored thereon instructions which, when executed, cause a computing device to perform the method of any one of claims 17-20.

Images