Cache control aware memory controller

Abstract

Systems, devices, and methods for performing efficient memory access for a computing system are disclosed. An external system memory is used as a last level cache and includes one of multiple types of dynamic random access memory (DRAM). A memory controller generates a tag request and a separate data request based on the same single received memory request. The tag request is sent prior to sending the data request. A partial tag comparison is performed during the processing of the tag request. If a tag miss is detected for the partial tag comparison, the data request is canceled and the memory request is sent to main memory. If one or more tag hits are detected for the partial tag comparison, the processing of the data request depends on the result of the full tag comparison.

Description

Background Art

[0001] Description of Related Technology

[0002] As semiconductor manufacturing processes develop and on-chip geometries decrease, semiconductor chips provide more functionality and performance. However, design issues that can limit potential benefits still occur in modern technologies for processing and integrated circuit design. One problem is that the interconnection latency per unit length continues to increase in successive generations of two-dimensional planar layout chips. In addition, the high electrical impedance between the individual chips can increase latency. In addition, due to the increase in parasitic capacitance on these longer signal paths, the signal that traverses from off-chip to another die may significantly increase the power consumption of these signals (e.g., increase 10 to 100 times). Therefore, the memory access latency for system memory (such as off-chip dynamic random access memory (DRAM)) is considerable.

[0003] Another design issue is that most software applications that access large amounts of data are typically memory-bound, since computation time is often determined by memory bandwidth. Memory access latency for off-chip dynamic random access memory (DRAM) can range from hundreds to more than a thousand clock cycles, and the increase in the number of cores in processor designs exacerbates the memory bandwidth issue.

[0004] In view of the above, efficient methods and systems for performing efficient memory access for a computing system are desired. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The advantages of the methods and mechanisms described herein may be better understood by referring to the following description in conjunction with the accompanying drawings, in which:

[0006] Figure 1 is a block diagram of one embodiment of a computing system.

[0007] Figure 2 is a block diagram of one implementation of a memory controller.

[0008] Figure 3 is a flow chart of one embodiment of a method for sending a memory access request to a system memory.

[0009] Figure 4 is a flow chart of one embodiment of a method for processing a tag access request to a cache storage arrangement in a system memory.

[0010] Figure 5 is a flow chart of another embodiment of a method for processing data access requests independently of tag access requests to a cache storage arrangement in system memory.

[0011] Although the present invention is susceptible to various modifications and alternative forms, specific embodiments are shown in the drawings and described in detail herein by way of example. However, it should be understood that the drawings and detailed description thereof are not intended to limit the present invention to the specific forms disclosed, but on the contrary, the present invention will cover all modifications, equivalents and alternatives falling within the scope of the present invention as defined by the appended claims. DETAILED DESCRIPTION

[0012] In the following description, many specific details are set forth to provide a thorough understanding of the methods and mechanisms presented herein. However, it will be appreciated by those of ordinary skill in the art that various embodiments may be practiced without these specific details. In some cases, well-known structures, components, signals, computer program instructions, and techniques are not shown in detail to avoid confusion with the methods described herein. It should be understood that for simplicity and clarity of illustration, the elements shown in the accompanying drawings are not necessarily drawn to scale. For example, the size of some elements may be enlarged relative to other elements.

[0013] Various systems, devices, methods, and computer-readable media for performing efficient memory access of a computing system are disclosed. In one embodiment, a computing system includes one or more computing resources and an external system memory. Examples of computing resources include a general-purpose central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), etc. The computing resource generates a memory access request. In various embodiments, a given computing resource includes a cache memory subsystem. When the given computing resource determines that the memory access request does not hit within the cache memory subsystem, the given computing resource sends the memory access request to a given memory controller.

[0014] A given memory controller is coupled to a memory device associated with a target address of a memory access request. If the computing system includes multiple memory devices, the operating system allocates address space between the multiple memory devices. The memory controller supports mapping between a request address of a computing resource and an address pointing to a storage location in one or more memory devices. In some embodiments, the memory controller is connected to a single memory device. In other embodiments, the memory controller is connected to multiple memory devices. In various embodiments, the memory controller includes one or more queues for storing memory access requests. In one embodiment, the memory controller includes a read queue for storing memory read requests and a separate write queue for storing memory write requests. In other embodiments, the memory controller includes a unified queue for storing both memory read requests and memory write requests.

[0015] In yet another embodiment, the memory controller includes one or more queues for storing received memory access requests and a separate queue for storing scheduled memory access requests selected from the one or more queues. Thus, the memory controller supports sending memory access requests to the memory device out of order based on priority and target address. In some embodiments, the memory controller also supports sending responses to the computing resources out of order based at least on priority.

[0016] The external system memory is implemented in one of a variety of types of dynamic random access memory (DRAM). DRAM provides at least a gigabyte of integrated memory within the microprocessor package. In various embodiments, a relatively large amount of row-based memory storage is used as a last level cache (LLC) before accessing system memory implemented using more DRAM and / or off-chip disk storage or solid-state memory. In various embodiments, a memory controller coupled to the DRAM divides a received single memory access request into two separate memory access requests. In one embodiment, the memory controller generates a tag access request based on the received memory access request, and generates a data access request different from the tag access request based on the received memory access request.

[0017] In some embodiments, the memory controller stores an indication that specifies that each of the tag access request and the data access request is generated based on the same received memory access request. In one embodiment, the indication is stored in each of the tag access request and the data access request. In some embodiments, each of the read queue and the write queue is divided into a tag queue and a separate data queue. In one embodiment, the indication is a unique identifier generated based on one or more of the following: an identifier generated based on a free list, a thread or process identifier, a computation source identifier, a target address, an age or timestamp, etc.

[0018] In various embodiments, the memory controller blocks sending a data access request until information generated by processing the tag access request is available. Thus, if no data access request is sent at all, the tag access request is sent to the memory device before the data access request. A given row of a plurality of rows within the memory device is read. The given row is pointed to by a target address in the tag access request. For example, a page index within the target address is used to identify the given row. In one embodiment, the given row stores one or more tags within a portion of the given row.

[0019] The memory controller compares only a portion of the one or more tags stored in the given row with only a portion of the tags in the target address of the tag access request. In various embodiments, the partial tag comparison has a significantly smaller latency than the full tag comparison. If a match is not detected, a tag miss occurs, and the memory controller cancels sending the data access request to the memory device. The memory controller invalidates or otherwise deallocates the queue entry storing the data access request. Thus, it is known relatively early that the requested data is not in the last level cache implemented in DRAM. For example, it is known after a significantly smaller latency of a partial tag comparison than after a significantly larger latency of a full tag comparison.

[0020] For tag misses, the memory controller also sends the single memory access request to off-chip system memory and / or main memory, such as disk storage or solid-state storage. In one embodiment, the memory controller maintains a copy of the single memory access request in a queue. In other embodiments, the memory controller regenerates the single memory access request based on one or more of the tag access request and the separate data access request.

[0021] In one embodiment, when the memory controller determines that one or more tag hits occur based on the partial tag comparison, the memory controller compares the entirety of the one or more tags in a given row with the entirety of the tags in the target address of the tag access request. In various embodiments, the full tag comparison has a significantly longer latency than the partial tag comparison. If the full tag comparison results in a tag miss, the memory controller performs the above steps for determining a tag miss based on the partial tag comparison. If the full tag comparison results in a tag hit, the memory controller sends a data access request to the memory device.

[0022] The memory controller performs a read access or a write access to the data in the given row based on the access type of the data access request. In one embodiment, the memory controller identifies the data in the given row to be accessed based on the position of the tag corresponding to the tag hit within the given row, the position indicating the position of the data within the given row.

[0023] See also Figure 1, a generalized block diagram of one embodiment of a computing system 100 is shown. As shown, the computing system 100 includes a communication structure 120 between each of the memory controllers 130 and the computing resources 110. In the illustrated embodiment, the computing resources include a central processing unit (CPU) 112, a graphics processing unit (GPU) 114, and a hub 116. The hub 116 is used to communicate with a multimedia engine 118. Although a single memory controller 130 is shown, in other embodiments, another number of memory controllers are used in the computing system 100. The memory controller 130 receives memory access requests from the computing resources 110 through the communication structure 120 and sends the memory access requests to one or more of an off-chip main memory 162 and a system memory implemented as a dynamic random access memory (DRAM) 170. The memory controller 130 also receives responses from the DRAM 170 and the main memory 162, and sends the responses to the corresponding sources of the requests in the computing resources 110.

[0024] In some embodiments, the components of computing system 100 are single dies on an integrated circuit (IC) such as a system on a chip (SOC). In other embodiments, the components are single dies in a system-in-package (SiP) or a multi-chip module (MCM). In one embodiment, computing system 100 is a stand-alone system in: a mobile computer, a smart phone, a smart watch, or a tablet; a desktop computer; a server; or other items. CPU 112, GPU 114, and multimedia engine 118 are examples of computing resources that can generate memory access requests. Although not shown, in other embodiments, other types of computing resources are included in computing resources 110.

[0025] Each of the one or more processor cores in CPU 112 includes a circuit for executing instructions according to a given selected instruction set architecture (ISA). In various embodiments, each of the processor cores in CPU 112 includes a superscalar, multithreaded micro-architecture for processing instructions of a given ISA. In one embodiment, GPU 114 includes a high parallel data micro-architecture with a large number of parallel execution channels. In one embodiment, micro-architecture uses single instruction multiple data (SIMD) pipelines for parallel execution channels. Multimedia engine 118 includes a processor for processing audio data and visual data of multimedia applications.

[0026] In one example, an accelerated processing unit (APU), display controller, audio processor, etc. are additional candidates to be included in computing resources 110. An example of an APU is a CPU that is integrated on the same die with a GPU, FPGA, or other processing unit, thereby increasing the data transfer rate between these units while reducing power consumption. In other embodiments, the APU includes video processing and other specialized accelerators.

[0027] In various embodiments, communication fabric 120 transfers traffic back and forth between computing resources 110 and memory controller 130 and includes interfaces for supporting corresponding communication protocols. In some embodiments, communication fabric 120 includes at least queues for storing requests and responses, selection logic for arbitrating between received requests before sending them across an internal network, logic for constructing and decoding packets, and logic for selecting routes for packets.

[0028] In some embodiments, the address space of computing system 100 is divided between at least CPU 112, GPU 114 and hub 116 and one or more other components such as input / output peripherals (not shown) and other types of computing resources. A memory map is maintained to determine which address is mapped to which component, and therefore to which of CPU 112, GPU 114 and hub 116 a memory request for a particular address should be routed. As software applications access more and more data, memory subsystems are increasingly utilized. Latency becomes more critical. More on-chip memory storage devices are used to reduce interconnect latency. For example, one or more of the computing resources within computing resource 110 includes a cache memory subsystem to reduce memory latency of the corresponding processor core. In various embodiments, a relatively large amount of row-based memory storage devices in DRAM 170 are used as a last level cache (LLC) before accessing off-chip system memory (not shown), which can be more DRAM or main memory 162 as disk storage or solid-state storage.

[0029] In various embodiments, DRAM 170 stores each bit of data in a separate capacitor within an integrated circuit. The capacitor can be charged or discharged. These two states are used to represent the two logical values ​​of the bit. DRAM 170 uses a single transistor and capacitor per bit. Compared to the six transistors used in on-chip synchronous RAM (SRAM), DRAM achieves higher density. Unlike HDD and flash memory, DRAM 170 is a volatile memory rather than a non-volatile memory. DRAM 170 loses its data relatively quickly when power is removed.

[0030] Recently, progress has been made in three-dimensional integrated circuits (3D ICs) that include two or more layers of active electronic components integrated vertically and horizontally into a single circuit. 3D packaging, known as system-in-package (SiP) or chip-stacked multi-chip modules (MCM), saves space by stacking individual chips in a single package. The components within these layers communicate using on-chip signaling (vertically or horizontally). This signaling provides shortened interconnect signal latency over known two-dimensional planar layout circuits.

[0031] In one embodiment, DRAM 170 includes a multi-channel memory architecture. This type of architecture increases the speed at which data is transferred to memory controller 130 by adding more communication channels therebetween. In one embodiment, the multi-channel architecture utilizes multiple memory modules and a motherboard and / or a card capable of supporting multiple channels. In another embodiment, as previously described, DRAM 170 utilizes a three-dimensional integrated circuit (3D IC) to provide the last level cache. Other memory technologies for the last level cache that may exist and are contemplated to use a row-based access scheme that includes one or more row buffers or other equivalent structures. Examples of other memory technologies include phase change memory, spin torque transfer resistive memory, memristor, etc.

[0032] As previously described, in various embodiments, DRAM 170 is used as a last level cache (LLC) before accessing system memory. In some embodiments, memory controller 130 includes control logic for processing RAM transactions such as DRAM open / activate, precharge, close, and refresh commands. In addition, memory controller 130 includes control logic for processing cache accesses such as tag comparisons and identifying a portion of a larger data read out based on tag comparisons. In various embodiments, DRAM 170 is filled with data from one or more of system memory (not shown) and main memory 162 through I / O controller and bus 160 and memory bus 150. The corresponding cache fill line with the requested block is transferred from DRAM 170 to the corresponding cache memory subsystem in the cache memory subsystem in computing resource 110 to complete the original memory access request. The cache fill line is placed in one or more levels of cache.

[0033] One example of a protocol for the corresponding interface of the memory controller 130 is a double data rate (DDR) type protocol. The protocol determines values ​​for information transfer, such as the number of data transfers per clock cycle, signal voltage levels, signal timing, signal and clock phases, and clock frequency. Protocol examples include DDR2 SDRAM, DDR3 SDRAM, GDDR4 (Graphics Double Data Rate, Version 4) SDRAM, and GDDR5 (Graphics Double Data Rate, Version 5) SDRAM. The memory controller 130 includes control circuits for interfacing with the memory channels and following the corresponding protocols.

[0034] As shown, DRAM 170 includes a plurality of memory array banks 174A to 174B. Each bank in bank 174A to 174B includes a corresponding bank in row buffer 172A to 172B. Each bank in row buffer 172A to 172B stores data in the accessed row in the multiple rows in memory array bank 174A to 174B. The accessed row is identified by the DRAM address in the received memory access request. The control logic in logic 138 of DRAM 170 performs complex transactions, such as activating (opening) transactions and precharging the data and control lines in DRAM 170, once to access the identified row, and once to put the modified content stored in the bank buffer back to the identified row during the closing transaction. Each of different DRAM transactions (such as activating / opening, column access, read access, write access, and precharge / close) has different corresponding waiting times. Typically, activating and precharge transactions have significantly higher waiting times than read access and write access transactions.

[0035] In various embodiments, DRAM 170 includes one or more memory channels, one or more memory modules or devices per channel, one or more banks per memory module, one or more memory banks per bank, and one or more rows per memory bank. Typically, each row stores a page of data. The size of the page is selected based on design considerations. Based on design choices, a page can be 1 kilobyte (1KB), 4 kilobytes (4KB), or any size.

[0036] A cache memory subsystem (not shown) in computing resource 110 includes a high-speed cache memory configured to store blocks of data. As used herein, a "block" is a group of bytes stored in a contiguous memory location that is considered a unit for consistency purposes. As used herein, each of the terms "cache block," "block," "cache line," and "line" are interchangeable. In some embodiments, a block may also be a unit of allocation and deallocation in a cache. The number of bytes in a block varies according to design choice and may be of any size. In addition, each of the terms "cache tag," "cache line tag," and "cache block tag" are interchangeable. In various embodiments, the cache memory subsystem is implemented as a hierarchy of caches.

[0037] As used herein, the term "access" refers to the performance of a memory read request or memory write request operation that results in a cache hit if the requested data corresponding to the request address resides in the cache. Alternatively, if the requested data does not reside in the cache, the memory access request results in a cache miss. As used herein, a "memory read request" is also referred to as a read request, and similarly, a "memory write request" is also referred to as a write request.

[0038] If the requested data is not found in the cache memory subsystem in the computing resource 110, a memory access request is generated and transmitted to the memory controller 130. As shown, the memory controller 130 includes a request queue 132 for queuing memory access requests received from the computing resource 110 via the communication structure 120. The memory controller 130 also has a response queue 134 for storing responses received from the DRAM 170. In one embodiment, the request queue 132 includes one or more queues for storing memory access requests. In another embodiment, the request queue 132 includes a read queue for storing memory read requests and a separate write queue for storing memory write requests. In other embodiments, the request queue 132 includes a unified queue for storing both memory read requests and memory write requests.

[0039] In yet another embodiment, request queue 132 includes one or more queues for storing received memory access requests and a separate queue for storing scheduled memory access requests selected from the one or more queues. Scheduler 136 includes one or more scheduling blocks for selecting memory access requests stored in request queue 132 to be sent out of order to DRAM 170. Thus, memory controller 130 supports sending memory access requests out of order to DRAM 170 based on priority and target address.

[0040] In various embodiments, the memory bus 150 supports sending data traffic in a single direction for a given amount of time (such as during a given mode of a read mode and a write mode), and then sends data traffic in the opposite direction for another given amount of time (such as during other modes of a read mode and a write mode). In one embodiment, the memory bus 150 utilizes at least a command bus and a data bus, and the memory bus 150 supports a read mode for sending data traffic on the data bus from the DRAM 170 to the memory controller 130. In addition, the memory bus 150 supports a write mode for sending data traffic on the data bus from the memory controller 130 to the DRAM 170.

[0041] In some embodiments, the memory controller 130 also supports out-of-order sending of responses to the computing resources 110 based at least on priority. In various embodiments, a scheduler 136 in the memory controller 130 schedules sending of stored memory access requests based on quality of service (QoS) or other priority information, age, process or thread identifier (ID), and relationship to other stored requests (such as to the same memory channel, to the same rank, to the same memory bank, and / or to the same page).

[0042] In various embodiments, the cache controller and memory device controller logic 138 (also referred to as logic 138) divides the received single memory access request into two separate memory access requests. In one embodiment, the logic 138 generates a tag access request based on the received memory access request, and generates a data access request different from the tag access request based on the received memory access request. The tag access request and the data access request may also be referred to as a cache tag access request and a cache data access request. In some embodiments, each of the read queue and the write queue within the request queue 132 is divided into a tag queue and a separate data queue. In one embodiment, the logic 138 generates a unique identifier based on one or more of the following: an identifier generated from a free list, a thread or process identifier, a computation source identifier, a target address, an age or timestamp, etc. In some embodiments, the logic 138 stores the generated unique identifier in each of the tag access request and the data access request.

[0043] In some embodiments, logic 138 schedules sending data access requests based on information generated by processing tag access requests. In various embodiments, logic 138 blocks sending data access requests until information generated by processing tag access requests is available. Therefore, if a data access request is not sent at all, the tag access request is sent to the memory device before the data access request is sent. In one embodiment, a page index within the target address is used to identify a given row of a plurality of rows in a memory bank targeted in memory banks 174A-174B. In one embodiment, a given row stores one or more tags within a portion of the given row. Since DRAM 170 is used as a last level cache such as a RAM cache memory, the memory device may be referred to as a RAM cache memory device.

[0044] In various embodiments, logic 138 performs a partial tag comparison between one or more tags stored in a given row and a tag in a target address of a single memory access request stored in a tag access request. In various embodiments, the partial tag comparison has a significantly smaller latency than a full tag comparison. If a match is not detected, a tag miss occurs, and logic 138 cancels sending the data access request to DRAM 170. The memory controller invalidates or otherwise deallocates the queue entry storing the data access request. Thus, it is known earlier that the requested data is not in the system memory, such as after a significantly smaller latency of the partial tag comparison than after a significantly larger latency of the full tag comparison. For a tag miss, logic 138 also sends a single memory access request to main memory 162. In some embodiments, the full tag comparison starts simultaneously with the partial tag comparison. If a match is not detected for the partial tag comparison, the full tag comparison is turned off or stopped, or the full tag comparison is allowed to continue, but the result will be discarded.

[0045] In one embodiment, when logic 138 determines that one or more tag hits occur based on the partial tag comparison, logic 138 compares the entirety of the one or more tags in a given row with the entirety of the tag in the target address of the tag access request. As previously described, in some embodiments, the full tag comparison is initiated simultaneously with the partial tag comparison. In various embodiments, this full tag comparison has a significantly longer latency than the partial tag comparison. If the full tag comparison results in a tag miss, logic 138 performs the above steps for determining a tag miss based on the partial tag comparison. If the full tag comparison results in a tag hit, logic 138 sends the data access request to DRAM 170. In one embodiment, the full tag hit result is used to identify a specific portion of the row that is later read out by processing of the data access request. For example, a specific cache line in the row is identified by the tag hit result.

[0046] Off-chip disk storage 162 can provide non-volatile, random access secondary storage of data. In one embodiment, off-chip disk storage 162 may include one or more hard disk drives (HDDs). HDDs typically include one or more rotating disks, each of which is coated with magnetic media. These disks rotate at a rate of several thousand revolutions per minute. Magnetic actuators are responsible for positioning the magnetic read / write devices on the rotating disks. In another embodiment, off-chip disk storage 162 utilizes a solid state disk (SSD). A solid state disk may also be referred to as a solid state drive. An SSD can simulate an HDD interface, but an SSD utilizes a solid state memory rather than an electromechanical device found in a HDD to store persistent data. For example, an SSD may include a memory bank of flash memory.

[0047] See now Figure 2 , shows a generalized block diagram of one embodiment of a memory controller 200. In the illustrated embodiment, the memory controller 200 includes an interface 210 to computing resources via a communication fabric, a queue 220 for storing received memory access requests and received responses, a control unit 250, and an interface 280 to memory devices via at least one data bus of a memory bus. Each of the interfaces 210 and 280 supports a corresponding communication protocol.

[0048] In one embodiment, queue 220 includes a read tag queue 232 and a read data queue 234 for storing generated requests based on received read requests. In one embodiment, queue 220 includes a write tag queue 236 and a write data queue 238 for storing generated requests based on received write requests. In one embodiment, queue 220 includes a queue 239 for storing scheduled memory access requests selected from one or more of queues 232-238 or a unified queue (if a unified queue is used).

[0049] In some embodiments, the read scheduler 252 includes arbitration logic for selecting read tag requests out of order from the read tag queue 232. The read scheduler 252 schedules sending the stored tag read requests within the read tag queue 232 out of order to the memory device based on quality of service (QoS) or other priority information, age, process or thread identifier (ID), and relationship to other stored requests (such as for the same memory channel, for the same rank, for the same memory bank, and / or for the same page). The write scheduler 254 includes similar selection logic for the write tag queue 236. In one embodiment, the response scheduler 256 includes similar logic for sending responses received from the memory device out of order to the computing resources based on priority.

[0050] In some embodiments, control register 270 stores an indication of the current mode. For example, the off-chip memory data bus and memory device support either a read mode or a write mode at a given time. Therefore, traffic is routed in a given single direction during the current mode and changes direction when the current mode is changed after the data bus turnaround latency. In various embodiments, control register 270 stores a threshold number of read requests (read burst length) to be sent during the read mode.

[0051] In one embodiment, the cache controller logic 260 divides the received single memory access request into two separate memory access requests. In one embodiment, the logic 260 generates a tag access request based on the received memory access request, and generates a data access request different from the tag access request based on the received memory access request. In some embodiments, the logic 260 stores an indication that specifies that each of the tag access request and the data access request is generated based on the same received memory access request. In one embodiment, the indication is stored in each of the tag access request and the data access request. In one embodiment, the indication is a unique identifier generated based on one or more of the following: an identifier generated based on a free list, a thread or process identifier, a computation source identifier, a target address, an age or timestamp, etc. In other embodiments, the logic 260 stores the indication in a corresponding queue entry, such as an entry in a read tag queue 232 that stores the contents of a read tag request and an entry in a read data queue 234 that stores the contents of a separate but corresponding read data request. The write tag queue 236 and the write data queue 238 store similar indications for write requests.

[0052] In various embodiments, the memory device logic 262 blocks sending a data access request until information generated by processing the tag access request is available. Thus, if a data access request is not sent at all, the tag access request is sent to the memory device before the data access request. In one embodiment, a page index within the target address is used by the logic 262 to identify a given row of a plurality of rows in a targeted bank of the memory device. In one embodiment, a given row stores one or more tags within a portion of the given row.

[0053] In various embodiments, logic 262 performs a partial tag comparison between one or more tags stored in a given row and a tag in a target address of a single memory access request stored in a tag access request. If a match is not detected, a tag miss occurs, and logic 262 cancels sending a data access request to the memory device. Logic 262 invalidates or otherwise deallocates a queue entry in a read data queue 234 or a write data queue 238 based on the access type. For a tag miss, logic 262 also sends a single memory access request to a separate system memory or main memory. In some embodiments, logic 262 begins performing a full tag comparison simultaneously with performing a partial tag comparison. If a match is not detected for the partial tag comparison, the full tag comparison is turned off or stopped, or the full tag comparison is allowed to continue, but the result will be discarded.

[0054] In one embodiment, when logic 262 determines that one or more tag hits occur based on the partial tag comparison, logic 262 compares the entirety of the one or more tags in the given row with the entirety of the tag in the target address of the tag access request. If the full tag comparison results in a tag miss, logic 262 performs the above steps for determining a tag miss based on the partial tag comparison. If the full tag comparison results in a tag hit, logic 262 sends the data access request to DRAM 170.

[0055] See now Figure 3 , shows one embodiment of a method 300 for sending a memory access request to a last level cache storage arrangement in DRAM. For discussion purposes, the embodiments (and Figures 4 to 5 However, it should be noted that in various embodiments of the described method, one or more of the described elements are performed simultaneously, in a different order than shown, or are omitted entirely. Other additional elements may also be performed as needed. Any of the various systems or devices described herein is configured to implement method 300.

[0056] One or more computing resources execute a computer program or software application. Examples of computing resources are given above. The computing resource determines that a given memory access request does not hit in a cache memory subsystem within the computing resource. The computing resource sends the memory access request to a system memory such as DRAM through a memory controller.

[0057] During execution of a computer program (block 302), a memory request that misses within a computing resource is detected (block 304). A given memory request is sent to a memory controller coupled to at least one memory device (block 306), where the given memory request is divided into a tag request and a separate data request (block 308). The tag request is scheduled to be sent to the memory device along with other tag requests (block 310). The data request is blocked from being sent until information generated by processing the tag request is available (block 312). The tag request selected by the scheduler is sent to the memory device (block 314). In some cases, the tag request is the only tag request because the given memory request is the only memory request. In this case, scheduling does not use information from multiple tag requests. In some embodiments, when there is only a single tag request and a single data request, if the memory request is a read request, the data request is speculatively sent along with the tag request. If a full tag miss eventually occurs, the retrieved data may be discarded. Otherwise, as further described in the upcoming description of methods 400 and 500, a partial tag comparison is performed before sending the data request.

[0058] See also Figure 4 , another embodiment of a method 400 for processing a tag access request to a last level cache storage arrangement in DRAM is shown. In the example shown, a tag request is sent to a memory device (block 402). As previously described, in various embodiments, logic for a cache controller (e.g., Figure 2 The cache controller logic then sends the tag request to the logic for the memory controller or the logic for the memory device (i.e., Figure 2 Logic 262 of the memory controller for the memory device sends a tag request to the memory device. In response to the tag request, a given row corresponding to the address in the tag request is identified in the DRAM cache (block 404). The given row is opened by an activation transaction (block 406). Information from the row is stored in a row buffer (block 408). A portion of one or more tags stored in the row buffer is compared to a portion of the tag in the address of the tag request (partial tag comparison) (block 410).

[0059] If a tag miss based on a partial tag comparison does not occur (the "no" branch of conditional block 412), a partial tag hit has occurred and the separate data request is processed (block 414). In one embodiment, the separate data request is a read request, and the data request is speculatively processed by sending the data request to the memory device. In some embodiments, a full tag comparison is also performed. In some embodiments, logic for the memory device performs a partial tag comparison, and logic for the cache controller performs a full tag comparison. If the data request is a write request, the data request is not processed until a full tag comparison is performed. The entirety of one or more tags stored in the line buffer is compared to the entirety of the tag in the target address of the tag request. In various embodiments, the latency of the full tag comparison is longer than the latency of the partial tag comparison. If a full tag miss is determined based on the full tag comparison, the data retrieved by sending the read request is discarded. If the data request is a read request and has been sent to the memory device, the retrieved data is discarded and the logic for the cache controller sends the corresponding memory request to the system memory. If the data request has not yet been sent, the logic for the cache controller cancels the data request and sends the corresponding memory request to the system memory. If a full tag hit is determined based on the full tag comparison, the data retrieved by sending the data request (read request) is sent to the logic for the cache controller to service the corresponding memory request. If a full tag hit occurs and the data request is a write request, the data request is sent to the memory device to service the memory request. Otherwise, if a tag miss is determined based on the partial tag comparison (the "yes" branch of conditional box 412), the corresponding data request sent to the memory device is canceled (box 416), and the memory request is sent to the system memory (box 418) to provide fill data in the DRAM cache. In various embodiments, the above-mentioned communication between the logic of the memory controller and the logic of the cache controller is used, and the above steps are used to discard data, cancel data requests, or service requests. In some embodiments, the logic for the memory device (i.e., Figure 2 The logic 262 of the cache controller sends an indication of a partial tag miss to the logic for the cache controller (ie, Figure 2 In another embodiment, the logic for the memory device discards the speculatively retrieved data and sends an indication of a partial tag miss to the logic for the cache controller.

[0060] See also Figure 5, another embodiment of a method 500 for processing a data access request independent of a tag access request to a last level cache storage arrangement in DRAM is shown. A partial tag hit is determined for a tag request (block 502). Subsequently, the entirety of one or more tags in the row buffer is compared to the entirety of the tag in the address of the tag request (block 504). If it is determined that a full tag miss has occurred (the "no" branch of conditional block 506), a memory request is sent to the system memory to provide fill data in the DRAM cache (block 508). As previously described, in various embodiments, if the data request is a read request and is speculatively sent to the memory device, the retrieved data is discarded and logic for the cache controller sends a corresponding memory request to the system memory. If the data request has not yet been sent, logic for the cache controller cancels the data request and sends the corresponding memory request to the system memory.

[0061] If it is determined that a full tag hit occurs (the "yes" branch of conditional block 506), the data request is sent to the memory device (block 510), and a given row corresponding to the address in the data request is identified within the DRAM cache (block 512). A read or write access to the corresponding cache line in the row buffer is performed based on the access type (block 514), the corresponding metadata is updated in the row buffer (block 516), and the row corresponding to the information in the row buffer is closed (block 518). As previously described, in some embodiments, when the data request is a read request, the data request is speculatively sent to the memory device based on a partial tag comparison before the full tag comparison is completed. In this case, if the full tag comparison determines that the full tag misses, the retrieved data is discarded. If the full tag comparison determines that the full tag hits, the retrieved data is provided earlier to service the read request.

[0062] In various embodiments, the previously described methods and / or mechanisms are implemented using program instructions of a software application. The program instructions describe the behavior of the hardware in a high-level programming language such as C. Alternatively, a hardware design language (HDL) such as Verilog is used. The program instructions are stored on a non-transitory computer-readable storage medium. Various types of storage media can be used. During use, a computing system can access the storage medium to provide the program instructions and accompanying data to the computing system for program execution. The computing system includes at least one or more memories and one or more processors configured to execute the program instructions.

[0063] It should be emphasized that the above embodiments are only non-limiting examples of implementations. Once the above disclosure is fully understood, many changes and modifications will become apparent to those skilled in the art. It is intended that the appended claims be interpreted as including all such changes and modifications.

Claims

1. A random access memory RAM cache memory controller, comprising: a first interface, the first interface being used to receive a memory access request; a second interface for sending cache access requests to a RAM cache memory device; as well as Control logic; In response to receiving a memory access request, the control logic is configured to: generating a cache tag access request based on the memory access request; generating a cache data access request different from the cache tag access request based on the memory access request; sending the cache data access request to the RAM cache memory device after information generated by processing the cache tag access request is available; sending the cache tag access request to the RAM cache memory device via the second interface; and Sending the cache data access request to the RAM cache memory device is scheduled based on information generated by processing the cache tag access request.

2. The RAM cache memory controller of claim 1 , wherein the control logic is further configured to: store a unique identifier in each of the cache tag access request and the cache data access request, wherein the unique identifier specifies that each of the cache tag access request and the cache data access request is generated from the same memory access request.

3. The RAM cache memory controller of claim 1 , wherein the control logic is further configured to: identifying within the RAM cache memory device a given row of a plurality of rows corresponding to an address in the cache tag access request; and Only a portion of one or more tags stored in the given line are compared to only a portion of the tags in the address of the cache tag access request.

4. The RAM cache memory controller of claim 3, wherein scheduling sending the cache data access request comprises: In response to determining a tag miss based on comparing the only a portion of the tags in the address, sending the cache data access request to the RAM cache memory device is cancelled.

5. The RAM cache memory controller of claim 3 , wherein in response to determining a tag miss based on comparing only a portion of the tag in the address, the control logic is further configured to send the memory access request to one or more of a system memory and a main memory.

6. The RAM cache memory controller of claim 3, wherein scheduling sending the cache data access request comprises: In response to determining a tag hit based on comparing an entirety of the one or more tags in the given row to an entirety of the tags in the address of the cache tag access request, sending the cache data access request to the RAM cache memory device through the second interface.

7. The RAM cache memory controller of claim 6, wherein the control logic is further configured to identify data in a given row accessed by the cache data access request based on a location of a tag corresponding to the tag hit resulting from an access by the cache tag access request.

8. A method comprising: generating, by a processor, a memory access request for data stored in a memory device; receiving, by a memory controller, the memory access request from the processor; as well as In response to receiving a memory access request: generating, by the memory controller, a cache tag access request based on the memory access request; generating, by the memory controller, a cache data access request different from the cache tag access request based on the memory access request; sending the cache data access request to a RAM cache memory device after information generated by processing the cache tag access request is available; sending, by the memory controller, the cache tag access request to the memory device; as well as Sending the cache data access request to the memory device is scheduled by the memory controller based on information generated by processing the tag access request.

9. The method of claim 8, further comprising: A unique identifier is stored by the memory controller in each of the cache tag access request and the data access request, wherein the unique identifier specifies that each of the cache tag access request and the cache data access request is generated from a same memory access request.

10. The method of claim 9, further comprising: identifying within the memory device a given row of a plurality of rows corresponding to an address in the tag access request; as well as Only a portion of one or more tags stored in the given row are compared to only a portion of the tags in the address of the tag access request.

11. The method of claim 10, wherein scheduling the sending of the cache data access request comprises: In response to determining a tag miss based on comparing the only a portion of the tags in the address, sending the cache data access request to the memory device is cancelled.

12. The method of claim 10, wherein in response to determining a tag miss based on comparing the only a portion of the tag in the address, the method further comprises sending the memory access request to one or more of a system memory and a main memory.

13. The method of claim 10, wherein scheduling the sending of the cache data access request comprises: In response to determining a tag hit based on comparing an entirety of the one or more tags in the given row to an entirety of the tags in the address of a cache tag access request, sending the cache data access request to the memory device.

14. The method of claim 13, further comprising: Data in a given row accessed by a cache data access request is identified based on a location of a tag corresponding to the tag hit resulting from an access by the cache tag access request.

15. A computing system comprising: a processor configured to generate a memory access request for data stored in a memory device; as well as A RAM cache memory controller as claimed in claims 1-7.

Images