Low Latency Cache

Abstract

The cache includes a plurality of sets, each set having a plurality of respective ways, and replacement logic configured to execute an LRU replacement policy based on a multi-stage LRU replacement calculation for a transaction. The multi-stages include a first stage in which the cache reads tag data for the transaction and makes a hit determination based on the tag data, a second stage in which the cache reads LRU data for the transaction, and a third stage in which the cache performs the LRU replacement calculation. If the hit determination is a hit, the cache is configured to provide the resulting cache data before completing the third stage.

Description

[Background technology]

[0001] The present disclosure relates to systems having integrated circuit devices. A cache is a device that stores data retrieved from or written to memory for one or more different hardware devices in a system. The hardware devices may be different components integrated into a system-on-chip (SOC). In this specification, devices that service read and write requests through a cache are referred to as client devices. Some caches service the memory requests of multiple different client devices integrated into a single system.

[0002] A cache can be used to reduce power consumption by reducing overall demands on main memory. Additionally, power can be further conserved by placing main memory and the data path to main memory in a low-power state as long as the client device can access the required data in the cache. Thus, cache usage correlates with overall power consumption, and increasing cache usage reduces overall power consumption. Thus, devices that rely on battery power, such as mobile computing devices, can extend battery life by increasing cache usage in integrated client devices.

[0003] A cache placement policy determines how memory blocks are placed in a cache. A set associative cache may use a least recently used (LRU) replacement policy. To implement an LRU replacement policy, the cache system needs to read the LRU data, which stores the most recently used information for cache lines in a set. Reading the LRU data can take time and can introduce latency into cache transactions. Summary of the Invention

[0004] This specification describes a cache system that implements a multi-stage, low-latency LRU replacement policy for transactions.

[0005] In one specific aspect of the present disclosure, a cache system is provided that includes a plurality of sets, each set having a plurality of respective ways, and replacement logic configured to execute an LRU replacement policy based on a multi-stage LRU replacement calculation for a transaction, the multi-stage including a first stage in which the cache reads tag data for the transaction and makes a hit determination based on the tag data, a second stage in which the cache reads LRU data for the transaction, and a third stage in which the cache performs the LRU replacement calculation, and if the hit determination is a hit, the cache is configured to provide the resulting cache data before the third stage is completed.

[0006] In some implementations of the cache system, if the hit determination is a miss, the cache is configured to issue a read request to memory before the third stage is complete.

[0007] In some implementations of the cache system, the multiple stages include an additional stage that occurs between the second and third stages, during which the cache waits to read the LRU data.

[0008] In some implementations of the cache system, the cache system includes a tag data table separate from the LRU data table, the tag data table configured to store tag data and the LRU data table configured to store LRU data.

[0009] In some implementations of the cache system, the tag data table and the LRU data table are implemented in separate memory devices.

[0010] In some implementations of the cache system, the replacement logic is configured to retain attribute data of two previous transactions, including a first transaction and a second transaction, where the first transaction precedes the second transaction and the second transaction precedes the current transaction, and the replacement logic is further configured to resolve a hazard condition upon a hit determination for the current transaction based on the attribute data.

[0011] In some embodiments of the cache system, to resolve the hazard condition, the cache system is configured to determine whether attribute data of a first transaction satisfies a first condition when the hit determination for the current transaction is a hit, the first condition indicating at least that the first transaction is scheduled to allocate an LRU way and the way information is still pending, and the cache system is further configured to determine the hit determination for the current transaction as pending in response to determining that the attribute data of the first transaction satisfies the first condition.

[0012] In some embodiments of the cache system, to resolve the hazard condition, the cache system is configured to determine whether the attribute data of the second transaction satisfies a second condition in response to determining that the attribute data of the first transaction does not satisfy a first condition, the second instruction indicating at least that the second transaction allocated an LRU way that matches the hit way of the current transaction, and the cache system is further configured to: determine a hit decision for the current transaction as a miss in response to determining that the second transaction satisfies the second condition; and determine a hit decision for the current transaction as a hit in response to determining that the second transaction does not satisfy the second condition.

[0013] In some implementations of the cache system, to resolve the hazard condition, the cache system is configured to determine whether attribute data of the first transaction satisfies a third condition when the hit determination for the current transaction is a miss, the third condition indicating at least that the first transaction has a tag portion with the same address as the current transaction and that the first transaction is to allocate an LRU way, and the cache system is further configured to determine the hit determination for the current transaction as a hit in response to determining that the attribute data of the first transaction satisfies the third condition.

[0014] In some embodiments of the cache system, to resolve the hazard condition, the cache system is configured to, in response to determining that the attribute data of the first transaction does not satisfy a third condition, determine whether the attribute data of the second transaction satisfies a fourth condition, the fourth instruction indicating at least that the second transaction has a tag portion with the same address as the current transaction and that the second transaction is to allocate an LRU way, and the cache system is further configured to, in response to determining that the second transaction satisfies the fourth condition, determine a hit determination for the current transaction as a hit, and in response to determining that the second transaction does not satisfy the fourth condition, determine a hit determination for the current transaction as a miss.

[0015] In another aspect of the present disclosure, there is provided a method, the method being performed by the cache system described above and including the operations described above.

[0016] The subject matter described herein can be implemented in particular embodiments to realize one or more advantages. For example, a cache system passes hit / miss results to downstream processing without waiting for the LRU data read to complete. This reduces the latency of cache transactions, thereby improving the time efficiency of the cache system. Furthermore, in some embodiments, the system stores tag data and LRU data in separate memory devices, thereby increasing design flexibility within cost or resource constraints.

[0017] The details of one or more embodiments of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. [Brief explanation of the drawings]

[0018] [Figure 1A] 1 shows an example of a cache system. [Figure 1B] 1 illustrates an example of a memory device that stores tag data and LRU data. [Figure 2] 1 is a flowchart illustrating an exemplary process for performing a cash transaction. [Figure 3] 1 illustrates an exemplary process for performing a cash transaction. [Figure 4A] 10 illustrates signal waveforms in an exemplary process for performing a cache transaction. [Figure 4B] 10 illustrates signal waveforms in another exemplary process for performing a cache transaction. [Figure 5A] 10 illustrates signal waveforms in an exemplary process for performing two cache transactions. [Figure 5B] 10 illustrates signal waveforms in another exemplary process for executing two cache transactions. [Figure 6]10 is a flowchart illustrating an exemplary process for performing hazard resolution for a current transaction. DETAILED DESCRIPTION OF THE INVENTION

[0019] Like reference symbols and designations in the various drawings refer to like elements. A cache placement policy determines how memory blocks are placed within a cache. Here, we focus on a set-associative cache placement policy, where the cache is divided into sets, each containing multiple cache lines.

[0020] 1A illustrates a cache system 100. The cache system 100 may be part of a processing system, such as a system-on-chip (SOC), communicatively coupled to a memory device. Specifically, the cache system 100 is a set-associative cache and includes multiple sets 110. Each set 110 includes multiple respective cache lines 114.

[0021] A cache line, also known as a way, is the unit of data transfer between a cache and another memory device (e.g., a processing system's main memory). All cache lines in a set have a fixed size, e.g., 64 bytes. The processor reads or writes the entire cache line when any location within the 64-byte region is read or written.

[0022] Cache system 100 further includes a cache transaction controller 120 that manages cache transactions for cache system 100. As used herein, a cache transaction refers to an access process to the cache system that involves a request for a specific memory block. An exemplary cache transaction process is described with reference to FIG. 2. Generally, cache transaction controller 120 maps a requested memory block to a particular set 110 using index bits derived from the memory block's address. Cache transaction controller 120 then performs a tag check to determine whether the requested memory block is already located in one of the cache lines 110. The tag check is performed based on tag data 130, which stores tags for all cache lines 114 in the memory device. Specifically, cache transaction controller 120 compares the tag bits of the memory block's address with the tags of the cache lines 114 in the mapped set 110. If the memory block's tag matches any cache line in the mapped set, the tag check returns a "cache hit." If there is no match, the tag check returns a "cache miss."

[0023] In the event of a "cache miss," cache transaction controller 120 requests a memory block from another memory device, such as the processing system's main memory or a next-level cache of the processing system, and places the memory block in a selected cache line 114 in mapped set 110. If all cache lines 114 in mapped set 110 are already allocated (i.e., their respective memory blocks have been previously placed), cache transaction controller 120 replaces the block stored in the identified cache line with new data read from the external memory device based on the replacement policy.

[0024] Specifically, cache transaction controller 120 uses cache replacement logic 125 to implement a least recently used (LRU) replacement policy that selects a least recently used cache line (among K ways) to replace. This process requires tracking the recency of use of each cache line 114 with respect to the usage of all other cache lines within a particular set 110. Accordingly, system 100 maintains LRU data 140 that specifies recency information for each cache line 114 within each set 110 of system 100.

[0025] In summary, cache transaction controller 120 controls system 100 and performs cache transactions in multiple stages, including reading the transaction's tag data 130, performing hit / miss determination, reading the transaction's LRU data 140, and performing LRU replacement calculations.

[0026] To improve the time efficiency of cache transactions, the system can adjust the start timeline of each operation in the process to minimize latency. An example operation timeline is described with reference to FIG. 3. Generally, to reduce the latency due to reading LRU data 140, if the hit determination is a hit, the cache system is configured to provide the obtained cache data before the execution phase of the LRU replacement calculation is completed. Furthermore, in some embodiments, if the hit determination is a miss, the cache system is configured to issue a read request to memory before the execution phase of the LRU replacement calculation is completed.

[0027] [Table 1]

[0028] In some implementations, cache system 100 further includes a data buffer 150 for storing attribute data of previous transactions. Cache transaction controller 120 uses the attribute data to resolve hazards during hit / miss determinations for the current transaction. An exemplary process for hazard resolution using attribute data is described in more detail with reference to FIG. 6.

[0029] Table 1 shows example data fields for tag data 130 and LRU data 140 for a particular cache line. When a large number of sets and cache lines exist in a cache system, tag data 130 and LRU data 140 can occupy a large amount of storage space. In some implementations, tag data 130 and LRU data 140 are stored separately from each other. For example, as shown in FIG. 1B , tag data 130 and LRU data 140 may be stored in two different memory devices, namely, first memory device 151 and second memory device 152. For example, first memory device 151 may be a first random access memory (RAM), and second memory device 152 may be a second RAM.

[0030] 1B, tag data 130 may be configured as a first table stored in a first memory device, and LRU data 140 may be configured as a second table stored in a second memory device 152. The first table may be an N×K table that stores tag information for N sets, each set having K cache lines. Each cell of the first table stores the tag of a particular cache line within a particular set. Similarly, the second table is also an N×K table that stores recency information, with each cell storing the recency of a particular cache line within a particular set.

[0031] Storing tag data 130 and LRU data 140 in separate memory devices allows for greater design flexibility within cost or resource constraints. A properly designed memory system will handle significantly more cache hits than cache misses during normal operation. Because cache hits do not require writing to the stored tag data, tag writes occur much less frequently than tag reads. In contrast, the frequency of LRU data reads and writes is more balanced. Therefore, to optimize the cost-performance ratio of a system design, tag data can be stored in a single-port RAM and LRU data can be stored in a dual-port RAM.

[0032] 2 illustrates an exemplary process 200 for performing a cache transaction. For convenience, process 200 is described as being performed by a cache system, such as cache system 100 of FIG. 1A.

[0033] Prior to executing process 200, the system has determined a set of cache transactions. The output of process 200 specifies a particular cache line, identified by an "access way" within the set, to access the memory block specified in the cache transaction request.

[0034] After receiving a cache transaction request that specifies a memory block, the system reads tag data in step 202 and reads LRU data in step 204. The system performs a tag check in step 210. Specifically, the system compares the tag bits associated with the address of the memory block with the tags of the cache lines in the set.

[0035] In step 210, the system determines whether the tag check is a cache hit or a miss. That is, if the tag bits associated with the address of the memory block match the tag of one of the cache lines in the set, the system determines that the tag check result is a cache hit. For convenience, a cache line having a tag that matches the memory block tag is referred to as a "hit way." If the tag bits associated with the address of the memory block do not match the tag of any of the cache lines in the set, the system determines that the tag check result is a miss.

[0036] If the system determines that the tag check result is a cache hit, the system assigns an access way to the hit way in step 230 to use the hit way for the transaction.

[0037] If the tag check results in a cache miss, meaning the data in the specified memory block is not loaded into any of the cache lines in the set, the system must request the data from the next level in the memory hierarchy and load the data into the selected cache line in the set.

[0038] The system checks whether there are any unused cache lines in the set in step 240. For convenience, unused cache lines are referred to as "freeways." If the system determines in step 240 that a freeway is available, then in a cache miss scenario, the system uses that freeway to execute the transaction. That is, the system assigns an access way to the freeway in step 260.

[0039] If, in step 240, the system determines that a freeway is not available, meaning that all cache lines in the set are in use, the system must identify cache lines to replace based on the replacement policy and place the data in the cache lines identified for replacement.

[0040] Since the system implements an LRU replacement policy, the system calculates the LRU way in step 270. This involves reading the LRU data and determining the LRU way based on the LRU data. The system assigns an access way to the LRU way in step 280.

[0041] Once the access way is determined (by step 230, 260, or 280), the system updates the LRU data in step 285 and then writes the LRU data to the second memory device in step 288. The system further updates the tag data in step 265 and writes the tag data to the first memory device in step 268.

[0042] To improve the time efficiency of process 200, the system can adjust the start timeline of each operation in the process to minimize latency. The step of calculating the LRU way (step 270) typically takes longer than the hit or miss decision (step 210). Step 270 also takes longer than the step of selecting a free way (step 240). As an example, the LRU way calculation may take two clock cycles to complete, while the hit / miss decision and free way selection may fit within one clock cycle. This is in part because the LRU data is stored in a second memory device, from which reading may take more time.

[0043] In some embodiments, to reduce the latency of cache transactions, when the cache transaction controller determines a hit / miss result in step 210, the system can pass the hit / miss decision to downstream processing without waiting for the LRU data read to be completed. For example, if the hit / miss decision is a hit, the system can start the process of retrieving cache data stored in the hit way and provide the resulting cache data to the transaction without waiting for the LRU data read to be completed. By starting the cache data retrieval early, the system can use the cache data from the hit way early, for example, before the LRU replacement calculation is completed. As another example, if the hit / miss decision is a miss, the system issues a read request to the next level in the memory hierarchy without waiting for the LRU data read to be completed. Because reading data from the next level in the memory hierarchy can take a considerable amount of time, this strategy can effectively reduce the latency of cache transactions.

[0044] 3 illustrates an exemplary timeline for performing operations including tag and memory-related operations 310 and LRU data-related operations 320 for a cache transaction, and shows how these operations fit within clock cycles. The timeline is divided into four consecutive time segments: T1, T2, T3, and T4. Each time segment may correspond to one or more clock cycles.

[0045] As shown in Figure 3, the system receives an incoming transaction during a first time segment T1 and issues a tag data read request to read the tag data of all cache lines that belong to the set associated with the received transaction. Table 2 shows example signals that are processed during the first time segment.

[0046] [Table 2]

[0047] In the second time segment T2, the system issues an LRU data read request to read the LRU data of all cache lines belonging to the set associated with the received transaction. Note that the LRU data read operation is delayed by one time segment compared to the tag read operation to account for the fact that it may take more time for the attribute data of the received transaction to be passed to the second memory device that stores the LRU data.

[0048] The tag data is available at T2. The system performs a tag check within this segment and then provides the hit / miss determination result as an output to subsequent processing logic. For example, if the hit / miss determination is a hit, the system can initiate the process of retrieving the cache data stored in the hit way and provide the resulting cache data to the transaction without waiting for the LRU data read to complete. In another example, if the hit / miss determination is a miss, the system can issue a read request to the next level in the memory hierarchy without waiting for the LRU data read to complete. These strategies can effectively reduce the latency of cache transactions. Table 3 shows examples of signals processed during the second time segment.

[0049] After the LRU data read request is issued, the system waits for the LRU data to become available in the third segment T3. After the LRU data becomes available through the read operation, the system provides the LRU data for LRU way calculation.

[0050] During the fourth segment T4, the system calculates the LRU way, updates the LRU data, and writes the updated LRU data to the second memory device. The system also updates the tag data and writes the updated tag data to the first memory device. The system also outputs data such as the final hit / miss status and way information. Table 4 shows an example of signals processed during the fourth time segment.

[0051] 4A shows the signal waveforms of the input signal (410A) of T1, the internal signal (415A), the output signal (420A) of T2, and the output signal (440A) of T4 when the hit / miss decision returns a hit in an exemplary process for executing a cache transaction. As shown in FIG. 4A, the system outputs a hit decision and a hit way at T2.

[0052] 4B shows signal waveforms of an input signal (410B) of T1, an internal signal (415B), an output signal (420B) of T2, and an output signal (440B) of T4 when a hit / miss decision returns a miss in an exemplary process for executing a cache transaction. As shown in FIG. 4B, at T2, the system outputs data indicating a miss decision and a "way pending" indication without waiting for the LRU data to become available at T4. At T4, the system obtains the LRU data and outputs the LRU way.

[0053] [Table 3]

[0054] [Table 4]

[0055] As described with reference to Figures 3, 4A, and 4B, by arranging operations in cache transactions, the system can reduce the latency caused by waiting for LRU data to become available. However, when determining the hit / miss outcome of the current transaction at T2, if a previous transaction overlaps with the current transaction in time (e.g., if the previous transaction is running at T3 or T4) and the overlapping previous transaction is mapped to the same set as the current transaction, a hazard may occur. This is because when a first transaction (e.g., transaction 1) results in a miss decision and proceeds to T4, the hit / miss indication data of two subsequent transactions (e.g., transaction 2 proceeding to T3 and transaction 3 proceeding to T2) mapped to the same set may depend on the access way of transaction 1, which may cause a hazard.

[0056] As an example, as shown in FIG. 5A, in a current transaction (e.g., transaction 3) that is determined to be a hit based on current tag data, the particular "hit way" determined based on the tag data may be an incorrect hit because a previous transaction (e.g., transaction 1) may have assigned that particular hit way to a different memory block.

[0057] As another example, as shown in FIG. 5B, in a current transaction (e.g., transaction 3) that is determined to be a miss based on the current tag data, the correct hit / miss decision may actually be a hit because the previous transaction (e.g., transaction 1) may have miss-allocated and may be waiting for LRU data to allocate the way.

[0058] The reason why the above hazard state occurs is that the latest tag data of the previous two transactions has not been updated at the time when the current transaction makes a hit / miss decision, and the LRU way of the previous transaction is still pending. If such a case is not detected or considered in T2 and only a tag lookup is performed, the hit / miss decision may be incorrect. Note that the hazard can only be caused by the "previous two transactions."

[0059] 6 is a flowchart illustrating an exemplary process for performing hazard resolution for a current transaction. For convenience, process 600 is described as being performed by a cache system, such as cache system 100 of FIG. 1A.

[0060] A hazard may occur when the current transaction makes a hit / miss determination in the T2 segment. To ensure reliable results in the hit / miss determination, the system needs to track attribute data for the two previous transactions. Therefore, the system maintains attribute data for the two previous transactions (shown in step 605). The two previous transactions include a first transaction and a second transaction, where the first transaction precedes the second transaction. That is, the first transaction is two time segments ahead of the current transaction, and the second transaction is one time segment ahead of the current transaction. When the current transaction advances to T2, the first transaction advances to T4, and the second transaction advances to T3. Table 5 shows an example of attribute data maintained for previous transactions.

[0061] In step 610, the system performs a tag check for the current transaction based on the current tag data. Note that the current data may still be updated by any previous transactions.

[0062] If the determination returns a hit, the system determines whether the attribute data of the first transaction satisfies a first condition in step 615. The first condition indicates that (1) the first transaction is valid, (2) the set index of the first transaction is the same as that of the current transaction, (3) the tag portion of the transaction address of the first transaction is different from that of the current transaction, and (4) the first transaction is to allocate an LRU way and the way information is still pending.

[0063] [Table 5]

[0064] If the system determines that the attribute data of the first transaction meets the first condition, the system determines the hit / miss status of the current transaction as pending in step 620. The system waits for the completion of the previous transaction to determine the final hit / miss status of the current transaction.

[0065] If the system determines that the attribute data of the first transaction does not satisfy the first condition, the system determines whether the attribute data of the second transaction satisfies a second condition in step 625. The second condition indicates that (1) the second transaction is valid, (2) the set index of the second transaction is the same as that of the current transaction, (3) the tag portion of the transaction address of the second transaction is different from that of the current transaction, and (4) the LRU way allocated by the second transaction matches the hit way of the current transaction.

[0066] If the system determines that the attribute data of the second transaction satisfies the second condition, the system determines the hit / miss status of the current transaction as a miss in step 635 .

[0067] If the system determines that the attribute data of the second transaction does not satisfy the second condition, the system determines the hit / miss status of the current transaction as hit in step 630 .

[0068] If the determination returns a miss, in step 640, the system determines whether the attribute data of the first transaction satisfies a third condition. The third condition indicates that (1) the first transaction is valid, (2) the set index of the first transaction is the same as that of the current transaction, (3) the tag portion of the transaction address of the first transaction is the same as that of the current transaction, and (4) the first transaction is to allocate an LRU way and the way information is still pending.

[0069] If the system determines that the attribute data of the first transaction satisfies the third condition, the system determines the hit / miss status of the current transaction as hit, but sets the way information as pending, in step 645. The system waits for the first transaction to determine the LRU way (of T4), and assigns the LRU way as the hit way of the current transaction.

[0070] If the system determines that the attribute data of the first transaction does not satisfy the third condition, the system determines whether the attribute data of the second transaction satisfies a fourth condition in step 650. The fourth condition indicates that (1) the second transaction is valid, (2) the set index of the second transaction is the same as that of the current transaction, (3) the tag portion of the transaction address of the second transaction is the same as that of the current transaction, and (4) the second transaction allocates the LRU way.

[0071] If the system determines that the attribute data of the second transaction satisfies the fourth condition, then in step 655, the system determines the hit / miss status of the current transaction as hit and assigns the hit way of the current transaction as the LRU way of the second transaction.

[0072] If the system determines that the attribute data of the second transaction does not satisfy the fourth condition, the system determines the hit / miss status of the current transaction as a miss in step 660 .

[0073] Because process 600 takes into account the attributes of the two previous transactions, it can properly resolve hazards when determining the hit / miss status of the current transaction.

[0074] As mentioned above, in order for the system to reduce the latency of cache transactions and improve the time efficiency of the cache system, the cache system can pass the hit / miss result to downstream processing without waiting for the LRU data read to complete. The system can execute a process to resolve any hazards that arise from initiating multiple transactions within consecutive clock cycles.

[0075] Embodiments of the subject matter and functional operations described herein may be implemented in digital electronic circuitry, in tangibly embodied computer software or firmware, in computer hardware, or one or more combinations thereof, including the structures disclosed herein and structural equivalents thereof. Embodiments of the subject matter described herein may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory storage medium for execution by or controlling the operation of a data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or one or more combinations thereof. Alternatively, or in addition, the program instructions may be encoded in an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal generated to encode information for transmission to a suitable receiver apparatus for execution by the data processing apparatus.

[0076] The term "configured" herein is used in reference to components of systems and computer programs. A system of one or more computers configured to perform a particular operation or action means that it has installed thereon software, firmware, hardware, or a combination thereof that, when run, causes the system to perform the operation or action. One or more computer programs configured to perform a particular operation or action means that the one or more programs contain instructions that, when executed by a data processing apparatus, cause the apparatus to perform the operation or action. Embodiments of the subject matter and functional operations described herein may be implemented in digital electronic circuitry, in tangibly embodied computer software or firmware, in computer hardware, or one or more combinations thereof, including the structures disclosed herein and structural equivalents thereof. Embodiments of the subject matter described herein may be implemented as one or more computer programs, i.e., as modules of one or more computer program instructions encoded on a tangible, non-transitory storage medium for execution by or controlling the operation of a data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or one or more combinations thereof. Alternatively or additionally, the program instructions may be encoded in an artificially generated propagated signal, such as a mechanically generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to a receiving device suitable for execution by a data processing device.

[0077] The term "data processing apparatus" refers to data processing hardware and encompasses all kinds of apparatus, devices, and machines for processing data, including, by way of example, a programmable processor, a computer, or multiple processors or computers. An apparatus can also be or further include special-purpose logic circuitry, such as an FPGA (field-programmable gate array) or an ASIC (application-specific integrated circuit). In addition to hardware, an apparatus can optionally include code that creates an execution environment for a computer program, such as code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof.

[0078] A computer program, referred to or described as a program, software, software application, app, module, software module, script, or code, can be written in any form of programming language, including compiled or interpreted, or declarative or procedural, and can be deployed in any form, including as a stand-alone program or as a module, or as components, subroutines, or other units suitable for use in a computing environment. A program may, but need not, correspond to a file in a file system. A program can be stored as part of a file that holds other programs or data, such as, for example, one or more scripts stored in a markup language document, or can be stored in a single file dedicated to the program, or in multiple cooperating files (e.g., files storing one or more modules, subprograms, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a data communications network.

[0079] The processes and logic flows described herein may be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by special purpose logic circuitry, such as an FPGA or an ASIC, or a combination of special purpose logic circuitry and one or more programmed computers.

[0080] A computer suitable for running a computer program may be based on a general-purpose or special-purpose microprocessor, or both, or on another type of central processing unit. Typically, the central processing unit receives instructions and data from a read-only memory, a random-access memory, or both. The essential elements of a computer are a central processing unit for implementing and executing instructions and one or more memory devices for storing instructions and data. The central processing unit and memory may be supplemented by, or incorporated in, special-purpose logic circuitry. Generally, a computer also includes one or more mass storage devices, such as magnetic, magneto-optical, or optical disks, for storing data, or is operatively coupled to receive data from or transmit data to them, or both. However, a computer need not have such devices. Furthermore, a computer may be incorporated in other devices, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), to name just a few.

[0081] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, by way of example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices, magnetic disks such as internal hard disks or removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks.

[0082] While the specification contains many specific implementation details, these should not be construed as limiting the scope of any invention or the scope of patentable subject matter, but rather as descriptions of features that may be unique to particular embodiments of a particular invention. Certain features described herein as separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented in multiple embodiments, either separately or in any suitable subcombination. Furthermore, even if features may be described above as functioning in a particular combination and originally claimed as such, one or more features from a claimed combination may, in some cases, be deleted from the combination, and the claimed combination may be directed to subcombinations or variations of the subcombinations.

[0083] Similarly, while operations are illustrated in the figures and described in the claims in a particular order, this should not be understood as requiring that such operations be performed in the particular order or sequential order shown, or that all of the operations shown be performed, to achieve desirable results. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the program components and systems described may generally be integrated into a single software product or packaged into multiple software products.

[0084] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims may be performed in a different order and still produce desirable results. As an example, the processes depicted in the accompanying figures do not necessarily require the particular order shown or sequential order to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. A cache, a plurality of sets, each set having a plurality of respective ways; replacement logic configured to implement an LRU replacement policy based on multiple stages of LRU replacement calculations for transactions, the multiple stages comprising: a first stage in which the cache reads tag data of the transaction and performs a hit determination based on the tag data; a second stage in which the cache reads the least recently used data of the transaction; a third stage in which the cache performs a least recently used (LRU) replacement calculation; If the hit determination is a hit, the cache is configured to provide the obtained cache data before the third stage is completed.

2. 2. The cache of claim 1, wherein if the hit determination is a miss, the cache is configured to issue a read request to memory before the third stage is complete.

3. 2. The cache of claim 1, wherein said multiple stages include an additional stage occurring between said second stage and said third stage, during which said cache waits to read said least recently used data.

4. 2. The cache of claim 1, wherein the cache includes a tag data table separate from an LRU data table, the tag data table configured to store tag data, and the LRU data table configured to store LRUs.

5. The cache of claim 4 , wherein the tag data table and the LRU data table are implemented in separate memory devices.

6. The substitution logic: configured to retain attribute data of two previous transactions, including a first transaction and a second transaction, the first transaction preceding the second transaction and the second transaction preceding a current transaction, the replacement logic further comprising: The cache of claim 1 , configured to resolve a hazard condition at the hit determination time for the current transaction based on the attribute data.

7. Resolving the hazard condition comprises: If the hit determination for the current transaction is a hit, determining whether the attribute data of the first transaction satisfies a first condition, the first condition indicating at least that the first transaction is to allocate a least recently used way and the way information is still pending; and resolving the hazard state further comprises:

7. The cache of claim 6, further comprising determining the hit determination for the current transaction as pending in response to determining that the attribute data of the first transaction satisfies the first condition.

8. Resolving the hazard condition further comprises: In response to determining that the attribute data of the first transaction does not satisfy the first condition, determining whether the attribute data of the second transaction satisfies a second condition, wherein the second indication at least indicates that the second transaction has allocated a least recently used way that matches a hit way of the current transaction; and resolving the hazard condition further comprises: In response to determining that the second transaction satisfies the second condition, determining the hit determination for the current transaction as a miss; and determining the hit determination for the current transaction as a hit in response to determining that the second transaction does not satisfy the second condition.

9. Resolving the hazard condition comprises: If the hit determination for the current transaction is determined to be a miss, determining whether the attribute data of the first transaction satisfies a third condition, the third condition indicating at least that the first transaction has a tag portion with the same address as the current transaction and that the first transaction is to allocate a least recently used way, and resolving the hazard state further comprises:

7. The cache of claim 6, further comprising determining the hit determination for the current transaction as a hit in response to determining that the attribute data of the first transaction satisfies the third condition.

10. Resolving the hazard condition further comprises: determining whether the attribute data of the second transaction satisfies a fourth condition in response to determining that the attribute data of the first transaction does not satisfy the third condition, wherein the fourth instruction indicates at least that the second transaction has the same tag portion of the address as the current transaction and that the second transaction is to allocate a least recently used way; and resolving the hazard condition further comprises: determining the hit determination for the current transaction as a hit in response to determining that the second transaction satisfies the fourth condition; and 10. The cache of claim 9, further comprising: in response to determining that the second transaction does not satisfy the fourth condition, determining the hit determination for the current transaction as a miss.

11. 1. A method of performing a least recently used (LRU) replacement calculation for a cache including a plurality of sets, each set having a plurality of respective ways, the method comprising: In a first stage, tag data of a transaction is read and a hit determination is performed based on the tag data; In a second phase, reading the least recently used data of the transaction; In a third step, performing an LRU replacement calculation; If the hit determination is a hit, providing the obtained cache data before completing the third step.

12. 12. The method of claim 11, further comprising issuing a read request to a memory before the third stage is completed if the hit determination is a miss.

13. 12. The method of claim 11, further comprising waiting to read the LRU data in an additional step occurring between the second step and the third step.

14. 12. The method of claim 11, further comprising storing tag data in a tag data table and storing LRU data in an LRU data table, said tag data table being separate from said LRU data table.

15. The method of claim 14 , wherein the tag data table and the LRU data table are implemented in separate memory devices.

16. The method further comprises: maintaining attribute data of two previous transactions, including a first transaction and a second transaction, the first transaction preceding the second transaction and the second transaction preceding a current transaction; The method of claim 11 , further comprising resolving the hit determination hazard condition for the current transaction based on the attribute data.

17. Resolving the hazard condition comprises: If the hit determination for the current transaction is a hit, determining whether the attribute data of the first transaction satisfies a first condition, the first condition indicating at least that the first transaction is to allocate a least recently used way and the way information is still pending; and resolving the hazard state further comprises:

17. The method of claim 16, further comprising determining the hit determination for the current transaction as pending in response to determining that the attribute data of the first transaction satisfies the first condition.

18. Resolving the hazard condition further comprises: In response to determining that the attribute data of the first transaction does not satisfy the first condition, determining whether the attribute data of the second transaction satisfies a second condition, wherein the second indication at least indicates that the second transaction has allocated a least recently used way that matches a hit way of the current transaction; and resolving the hazard condition further comprises: In response to determining that the second transaction satisfies the second condition, determining the hit determination for the current transaction as a miss; 20. The method of claim 17, further comprising: in response to determining that the second transaction does not satisfy the second condition, determining the hit determination for the current transaction as a hit.

19. Resolving the hazard condition comprises: If the hit determination for the current transaction is determined to be a miss, determining whether the attribute data of the first transaction satisfies a third condition, the third condition indicating at least that the first transaction has a tag portion with the same address as the current transaction and that the first transaction is to allocate a least recently used way, and resolving the hazard state further comprises:

17. The method of claim 16, further comprising determining the hit determination for the current transaction as a hit in response to determining that the attribute data of the first transaction satisfies the third condition.

20. Resolving the hazard condition further comprises: determining whether the attribute data of the second transaction satisfies a fourth condition in response to determining that the attribute data of the first transaction does not satisfy the third condition, wherein the fourth instruction indicates at least that the second transaction has the same tag portion of the address as the current transaction and that the second transaction is to allocate a least recently used way; and resolving the hazard condition further comprises: determining the hit determination for the current transaction as a hit in response to determining that the second transaction satisfies the fourth condition; and 20. The method of claim 19, further comprising: in response to determining that the second transaction does not satisfy the fourth condition, determining the hit determination for the current transaction as a miss.

Images