Time-efficient implementation of cache replacement policies

Abstract

The cache includes a plurality of sets, each set having a plurality of respective ways, and replacement logic configured to implement a two-phase LRU (Least Recently Used) replacement calculation that causes the cache to perform a first phase in which the cache calculates an LRU way for a set and a second phase in which the cache updates an LRU data structure with information about transactionally accessed ways.

Description

[Background technology]

[0001] The present disclosure relates to systems having integrated circuit devices.

[0002] A cache is a device that stores data retrieved from or written to memory for one or more different hardware devices in a system. The hardware devices may be different components integrated into a system on a chip (SOC). In this specification, devices that service read and write requests through a cache are referred to as client devices. Some caches service memory requests for multiple different client devices integrated into a single system.

[0003] A cache can be used to reduce power consumption by reducing overall requests to main memory. Additionally, power can be further conserved by placing main memory and the data path to main memory in a low-power state as long as the client device can access the required data in the cache. Thus, cache usage correlates with overall power consumption, and increasing cache usage reduces overall power consumption. Thus, devices that rely on battery power, such as mobile computing devices, can extend battery life by increasing cache usage in integrated client devices.

[0004] A cache placement policy determines how memory blocks are placed in a cache. For a set-associative cache, a least recently used (LRU) replacement policy can be used. In a traditional LRU replacement implementation, the system matches all cache replacement operations within the same clock cycle for a transaction. However, because the execution time of most of the operations of an LRU replacement policy increases with the number of cache lines in the cache's set, it can be difficult to fit all operations of a transaction into a single clock cycle for a highly associative set-associative cache. Summary of the Invention

[0005] This specification describes a cache system that includes multiple sets of cache lines and replacement logic configured to implement a two-phase LRU replacement policy. By performing LRU replacement in two phases, the cache system overcomes some limitations of traditional LRU replacement implementations and improves time efficiency for performing multiple cache transactions.

[0006] In one particular aspect of the present specification, a cache system is provided that includes a plurality of sets, each set having a plurality of respective ways, and replacement logic configured to implement a two-phase LRU replacement calculation that causes the cache to perform a first phase in which the cache calculates an LRU way for the set, and a second phase in which the cache updates an LRU data structure with information about transactionally accessed ways.

[0007] In some implementations of a cache system, the cache is configured to perform a two-stage LRU replacement calculation on successive clock cycles.

[0008] In some implementations of a cache system, the cache is configured to implement an LRU replacement policy using the calculated LRU way.

[0009] In some implementations of the cache system, the replacement logic includes a plurality of nodes that store respective way values ​​for each set, and performing the second stage includes shifting the stored way values ​​between the nodes for the set.

[0010] In some implementations of the cache system, to perform the second phase, the cache system is configured to update each node with a set that includes data that the node already stores or data shifted from other nodes.

[0011] In some implementations of the cache system, when executing the first phase for a current transaction, the cache system performs hazard resolution in response to a hazard condition resulting from the second phase of a previous transaction executed on the same particular set.

[0012] In some implementations of the cache system, performing hazard resolution for the current transaction includes determining an LRU way for the current transaction based on an LRU data structure after the LRU data structure has been updated for the previous transaction in response to determining that all of the respective ways in the particular set are eligible for replacement.

[0013] In some implementations of the cache system, the replacement logic is configured, in response to determining that not all of the respective ways in the particular set are eligible for replacement, to determine a plurality of eligible nodes from the plurality of nodes where the hazard has been resolved, select an eligible node from the plurality of eligible nodes where the hazard has been resolved, and determine an LRU way using a way value stored in the selected eligible node.

[0014] In some implementations of the cache system, the replacement logic is configured to begin determining eligible nodes from which the hazard has been resolved before the second phase of the previous transaction is completed.

[0015] In some embodiments of the cache system, determining a plurality of eligible nodes for which the hazard has been resolved from the plurality of nodes includes generating unresolved node data that identifies eligible nodes from the plurality of nodes based on the eligible ways and the LRU data structure before the LRU data structure was updated by the second phase of the previous transaction, and shifting the unresolved node data using shift information generated during the second phase of the previous transaction to generate hazard-resolved node data that identifies a plurality of eligible nodes for which the hazard has been resolved.

[0016] In another aspect herein, a method is provided for performing an LRU replacement calculation, the method being performed by the cache system described above and including the operations described above.

[0017] The subject matter described herein can be implemented in particular embodiments to achieve one or more advantages. For example, in some embodiments, a cache system improves time efficiency for executing multiple cache transactions by executing a two-stage LRU replacement policy in two consecutive clock cycles. Furthermore, in some embodiments, if a hazard occurs between a current transaction and a previous transaction, the system can initiate the hazard resolution process for the current transaction before the LRU update for the previous transaction finishes, minimizing the time impact of hazard resolution. Improved timing efficiency can be useful for implementing advanced LRU replacement policies in set-associative caches with high associativity.

[0018] The details of one or more embodiments of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. Like reference numbers and characters in the various drawings refer to like elements. [Brief explanation of the drawings]

[0019] [Figure 1] 1 shows an example of a cache system. [Figure 2] 1 is a flowchart illustrating an exemplary process for performing a cash transaction. [Figure 3] 1 illustrates an exemplary process for performing multiple cash transactions. [Figure 4A] 10 illustrates an exemplary data structure for LRU data for a set. [Figure 4B] 1 illustrates an exemplary process for updating LRU data. [Figure 5] 10 is a flowchart illustrating an example process for performing hazard resolution for a cash transaction. [Figure 6] 1 illustrates an exemplary hardware implementation of an LRU cache replacement policy. [Figure 7] 1 illustrates an exemplary hardware implementation of cache hazard resolution. DETAILED DESCRIPTION OF THE INVENTION

[0020] A cache placement policy determines how memory blocks are placed in a cache. In this specification, we focus on a set-associative cache placement policy, where the cache is divided into sets, each containing multiple cache lines.

[0021] 1 illustrates a cache system 100. The cache system 100 may be part of a processing system, such as a system-on-chip (SOC), communicatively coupled to a memory device. Specifically, the cache system 100 is a set-associative cache and includes multiple sets 110. Each set 110 includes multiple respective cache lines 114.

[0022] A cache line, also known as a way, is the unit of data transfer between a cache and another memory device (e.g., a processing system's main memory). All cache lines in a set have a fixed size, e.g., 64 bytes. The processor reads or writes the entire cache line when any location within a 64-byte region is read or written.

[0023] Cache system 100 further includes a cache transaction controller 120 that manages cache transactions for cache system 100. As used herein, a cache transaction refers to the process of accessing a cache system with a request for a particular memory block. An exemplary cache transaction process is described with reference to FIG. 2. Generally, cache transaction controller 120 maps a requested memory block to a particular set 110 using index bits derived from the memory block's address. Cache transaction controller 120 then performs a tag check to determine whether the requested memory block is already located in one of the cache lines 110. The tag check is performed based on tag data 130, which stores tags for all cache lines 114 in the memory device. Specifically, cache transaction controller 120 compares the tag bits of the memory block's address with the tags of the cache lines 114 in the mapped set 110. If the memory block tag matches any cache line in the mapped set, the tag check returns a "cache hit." Otherwise, the tag check returns a "cache miss."

[0024] In the event of a "cache miss," cache transaction controller 120 requests the memory block from another memory device, such as from the processing system's main memory or from a next-level cache of the processing system, and places the memory block in a selected cache line 114 of mapped set 110. If all cache lines 114 in mapped set 110 are already allocated (i.e., previously placed with their respective memory blocks), cache transaction controller 120 uses new data read from an external memory device to replace the block stored in the cache line identified by the replacement policy.

[0025] Specifically, system 100 uses cache replacement logic 125 to implement an LRU replacement policy that selects the least recently used cache line (out of K ways) for replacement. This process requires tracking the recency of each cache line 114 with respect to the usage of all other cache lines within a particular set 110. Accordingly, system 100 maintains LRU data 140 that specifies recency information for each cache line 114 within each set 110 of system 100.

[0026] Cache replacement logic 125 implements the LRU replacement calculation in two stages. During the first stage, cache system 100 determines the least recently used cache line (referred to in this application as the LRU way) based on at least current LRU data 140. The LRU way is selected for replacement. During the second stage, cache system 100 updates LRU data 140 based on information about the transaction-accessed way so that LRU data 140 is up-to-date for the next transaction.

[0027] 2 illustrates an exemplary process 200 for performing a cache transaction. For convenience, process 200 is described as being performed by a cache system, such as cache system 100 of FIG.

[0028] Prior to executing process 200, the system has determined a set for the cache transaction. The output of process 200 specifies a particular cache line within the set (referred to herein as an "access way") to access the memory block specified in the cache transaction request.

[0029] After receiving a cache transaction request that specifies a memory block, the system performs a tag check in step 210. Specifically, the system compares the tag bits associated with the address of the memory block with the tags of the cache lines in the set.

[0030] In step 220, the system determines whether the tag check results in a cache hit or a miss. That is, if the tag bits associated with the address of the memory block match the tag of one of the cache lines in the set, the system determines that the tag check result is a cache hit. For convenience, a cache line having a tag that matches the memory block tag is referred to as the "hit way." If the tag bits associated with the address of the memory block do not match the tag of any of the cache lines in the set, the system determines that the tag check result is a miss.

[0031] If the system determines that the tag check result is a cache hit, the system assigns an access way to the hit way in step 230 to use the hit way for the transaction.

[0032] If the system determines that the tag check result is a cache miss, this means that the data in the specified memory block is not loaded into any of the cache lines in the set, and the system must request the data from the next level of the memory hierarchy and load the data into the selected cache line in the set.

[0033] The system checks for cache lines in the unoccupied set in step 240. For convenience, unoccupied cache lines are referred to as "free ways." If the system determines that a free way is available in step 250, the system uses the free way to execute the transaction in the cache miss scenario. That is, the system assigns an access way to the free way in step 260.

[0034] If, in step 250, the system determines that a freeway is not available, meaning that all cache lines in the set are occupied, the system must identify cache lines for replacement through a replacement policy and place the data in the cache lines identified for replacement.

[0035] Since the system implements an LRU replacement policy, the system calculates the LRU way in step 270. In step 280, the system assigns an access way to the LRU way.

[0036] Once the access way is determined (by steps 230, 260, or 280), the system updates the LRU data in step 290. The LRU data tracks the recency of the cache lines in the set, i.e., defines how recently a particular cache line has been accessed for a transaction compared to other cache lines in the set. The update process is described in more detail with reference to Figures 4A-6.

[0037] In a conventional LRU replacement implementation, the system matches all of the operations shown in Figure 2 within the same clock cycle for a transaction. In this case, the time required to complete these operations can be estimated as follows: T conventional =O (Determine access way) +O (Update LRU data) = Max[O(determine hit way), O(select free way), O(calculate LRU way)] + O(update LRU data) + C (1)

[0038] Here, O(<operation>) denotes the timing depth to complete <operation>, and C is the constant time delay for an operation such as selecting one of the hit-way / free-way / compute-LRU-way branches shown in Figure 2.

[0039] Note that as the total number of cache lines in the set increases, each of the three terms, O (determine hit way), O (select free way), and O (calculate LRU way), increases. That is, as cache associativity increases, the time to determine the access way increases. When cache associativity is large (e.g., when the total number of cache lines in the set exceeds a threshold), it may become difficult to fit all operations of a transaction into a single clock cycle.

[0040] To overcome the limitations of conventional LRU replacement implementations, the system described herein performs LRU replacement using a two-stage scheme, i.e., the system implements a two-stage LRU replacement policy in successive clock cycles.

[0041] In the first phase, the system determines the access way, and in the second phase, the system updates the LRU data.

[0042] Therefore, the time required for the first clock cycle is: T1=O (Determine access way) =Max[O(determine hit way), O(select free way), O(calculate LRU way)] + C (2)

[0043] The time required for the second clock cycle is: T2=O (update LRU data) (3)

[0044] By dividing the time required to execute the operations of a transaction into two clock cycles, the system can generally improve the time efficiency for executing cache transactions. For example, after the system determines the access way in the first phase for a particular transaction, instead of waiting for the second phase to finish updating the LRU data for the particular transaction, the system can start the first phase for the next cache transaction. That is, while the second phase for a particular cache transaction is being executed, the system can start determining the access way for the next cache transaction. Thus, the second phase of the current transaction and the first phase of the next transaction can be performed in parallel. This scheme generally reduces the total time required to execute multiple cache transactions.

[0045] Figure 3 shows the clock cycle for performing cache operations for three cache transactions. The first transaction (txt1) is performed on the first set (set A) in the cache. The second transaction (txt2) is performed on the second set (set B) in the cache. The third transaction (txt3) is also performed on the second set (set B).

[0046] As shown in Figure 3, at clk=1, the system executes the first phase of the first transaction. At clk=2, the system executes the second phase of the first transaction and the first phase of the second transaction. At clk=3, the system executes the second phase of the second transaction and the first phase of the third transaction.

[0047] Note that at clk=2, the system executes two phases in parallel: the second phase of the first transaction and the first phase of the second transaction. This timing overlap does not create a hazard because the first and second transactions execute on different sets of the cache. That is, the first transaction executes on set A and the second transaction executes on set B. As used herein, hazard refers to a problem in the instruction pipeline when, for example, the next instruction cannot execute correctly in the next clock cycle due to an instruction exhibiting a data dependency on data modified in a different stage of the pipeline. Because the first and second transactions execute on different sets, there is no data dependency between the two transactions, and no hazard occurs.

[0048] In contrast, the timing overlap of the two phases at clk=3 may result in a hazard because both the second and third transactions are executed on the same set (set B). During the clk=3 cycle, the first phase of the third transaction may need to use LRU data from this set to determine the access way (e.g., to calculate the LRU way) while the LRU data from the same set is being updated in the second phase of the second transaction. This results in a hazard. Therefore, the hazard needs to be resolved in order to correctly calculate the LRU way for the third transaction. Exemplary processes and embodiments for resolving the hazard are described with reference to FIGS. 5 and 7.

[0049] 4A shows an example data structure 400 for storing LRU data. The data structure includes a sequence of K nodes with a one-to-one correspondence to the K cache lines in the set. Each node 410 stores a way value 412 that identifies the corresponding cache line. Each node 410 is associated with a respective node index, e.g., k=0, 1, ..., (K-1). The node index k of each node 410 specifies the recency of the cache line identified by the way value 412 of the node 410. For example, the cache line identified by the way value stored in the 0th node is the least recently accessed cache line in the set, and the cache line identified by the way value stored in the (k-1)th node is the most recently accessed cache line in the set.

[0050] During the process of updating the LRU data, ie, during the second phase of the cache transaction, the system updates the LRU data by shifting the way values ​​412 stored in the sequence 410 of the node.

[0051] FIG. 4B illustrates an exemplary process for updating LRU data stored in a data structure, for example, in the sequence of nodes shown in FIG. 4A. In FIG. 4B, the top row of nodes stores way values ​​before the LRU data update. The bottom row of nodes stores way values ​​after the LRU data update. Both rows of nodes are arranged in descending order of recency from left to right. That is, the leftmost node in each row stores the way value of the Most Recently Used (MRU) cache line, and the rightmost node in each row stores the way value of the Least Recently Used (LRU) cache line.

[0052] In the example shown in Figure 4B, the access way for the transaction is identified by the way value "2" stored in the fourth node from the left in the top row. The system updates the LRU data to reflect the change in recency by moving the way value "2" to the MRU node and shifting the way values ​​stored in the first through third nodes one node from left to right, resulting in the way value shown in the bottom row.

[0053] 5 illustrates an exemplary process 500 for performing hazard resolution in an LRU calculation. For convenience, process 500 is described as being performed by a cache system, such as cache system 100 of FIG.

[0054] When the system begins calculating the LRU way for the current transaction, if the current transaction is mapped to the same set to which the previous transaction is mapped and the previous transaction started one clock cycle before the current transaction, a data hazard may occur and the system will execute process 500 to resolve the hazard.

[0055] As an illustrative example, referring to FIG. 3, if transaction #3 (txn3) is the current transaction executing in set B, a data hazard may occur between the calculation of the LRU way (T1 phase) of txn3 and the update of the LRU data (T2 phase) of a previous transaction (txn2) also executing in set B.

[0056] Referring back to FIG. 5 , in step 510 of hazard resolution process 500, the system determines whether all cache lines in the set are eligible for replacement. This step is necessary because, in some scenarios, a cache system may be accessible by multiple client devices. As used herein, a client device is a device that issues read and write requests to the cache system to access data in the cache system. When a cache system is being accessed by multiple client devices, each client device may use only a subset of the cache lines in its current set for replacement. Thus, for a current transaction associated with a particular client device, only a subset of the cache lines are eligible for replacement, and the system may calculate the LRU way only from the eligible cache lines.

[0057] If the system determines that all cache lines in the set are eligible for replacement, the system need only perform step 520 and wait for the LRU updates to finish, and then perform step 525 to determine the LRU way for the current transaction based on the updated LRU data (which has been updated for the previous transaction). For example, in step 525, the system may simply determine the LRU way specified by the way value 412 stored in the LRU node (node ​​0) of the LRU data 400 as shown in Figure 4A, where all LRU data nodes 410 have been updated for the previous transaction. Because the LRU way is selected based on the latest LRU data for the previous transaction, the selected LRU way resolves the hazard.

[0058] As an illustrative example, referring to FIG. 3, to resolve a hazard between a current transaction (txn3) and a previous transaction (txn2), when all cache lines in set B are eligible for replacement, the system can simply wait for the update of the LRU data for txn2 in clk=3 to finish before starting the calculation of the LRU way for txn3.

[0059] 5, if the system determines that not all cache lines in a set are eligible for replacement, i.e., only a subset of the cache lines in the set are eligible for replacement for the current transaction, the system needs to identify the eligible subset of cache lines before determining the LRU way from the eligible cache lines. Because cache line recency information can be stored in each LRU data node (e.g., as shown in FIG. 4A), the system can determine eligible cache lines by determining a set of eligible LRU data nodes and identifying eligible cache lines using the way values ​​412 stored in each eligible LRU data node. Here, eligible LRU data nodes refer to the LRU data nodes that store the way values ​​that identify eligible cache lines.

[0060] Because process 500 is executed for the current transaction during the same clock cycle that the previous transaction is updating the LRU data, to correctly identify the eligible way, the system must determine the eligible nodes from which the hazard has been resolved in step 530. Here, the eligible nodes from which the hazard has been resolved refer to the LRU data nodes that store the latest way value that identifies the eligible cache line. The latest way value refers to the way value that is consistent with the updated LRU data from the previous transaction.

[0061] In some implementations, to improve time efficiency, the system begins determining eligible nodes for which hazards have been resolved for the current transaction before completing the LRU data update for the previous transaction. This can be achieved by directly using the node shift information that updates the LRU data in the previous transaction. The node shift information specifies, for each node in the current set of LRU data, that the node has been shifted (or will be shifted) by the LRU data update process of the previous transaction. An exemplary implementation of determining eligible nodes for which hazards have been resolved in a time-efficient manner is described with reference to FIG. 7.

[0062] After determining the eligible nodes whose hazards have been resolved in step 530, the system selects an LRU node from the eligible nodes whose hazards have been resolved in step 540. The selected LRU node is the least recently used node in the set of eligible nodes whose hazards have been resolved. In step 550, the system determines the LRU way using the way value stored in the selected LRU node.

[0063] Figure 6 illustrates an exemplary hardware implementation 600 of an LRU cache replacement policy applied to LRU data (i.e., each way value of a cache line in a set) stored in a sequence of nodes 640. Similar to the data structure illustrated in Figure 4, the sequence of K nodes 610 is arranged according to the recency (decreasing recency from left to right) of the cache lines identified by each way value stored in the node. That is, in the example illustrated in Figure 6, the leftmost node, i.e., node (K-1), stores the way value of the MRU cache line, and the rightmost node, i.e., node 0, stores the way value of the LRU cache line.

[0064] Circuit 600 includes a plurality of comparators 640, a plurality of OR gates 650, and a plurality of multiplexers 660. The circuit receives as input an access way 605 determined by the first phase of a cache transaction and compares access way 605 with each of the way values ​​stored in node 610 using respective comparators 640. The outputs of comparators 640 are connected to first inputs to each of the logical OR gates 650.

[0065] For ease of explanation and without loss of generality, the way value stored in node k is compared with the access way 605 using the kth comparator 640, where k=0, 1, 2, ..., (K-1). An OR gate exists for the kth node, where k=1, 2, 3, ..., (K-1). If k>1, the output of the kth comparator 640 is connected to the first input of the kth OR gate, and the second input of the kth OR gate is connected to the output of the (k-1)th OR gate. If k=1, the output of the kth comparator 640 is connected to the first input of the kth OR gate, and the second input of the kth OR gate is connected to the output of the (k-1)th comparator. That is, the OR gate performs an OR operation on the comparison results of the current node (node ​​k) and all preceding nodes (nodes 0, 1, ..., k-1). If one of nodes 0, 1, ..., k stores a way value that matches the access way 605, the kth OR gate outputs "1". If none of the nodes 0, 1, . . . , k stores a way value that matches the access way 605, the k-th OR gate outputs "0".

[0066] The output of the kth OR gate is connected to the control input of a respective multiplexer 660 (MUX k), where k = 1, 2, ..., (K-1). The control input of the rightmost multiplexer (MUX0) is connected to the comparison result of node 0. The data input of MUX k is connected to the current node (node ​​k) and the next node (node ​​(k+1)), with k = 0, ..., (K-2), respectively. The data input of the leftmost multiplexer (MUX(k-1)) is connected to the input of node (K-1) and the access way, respectively. Thus, multiplexer 660 outputs updated LRU data 630, i.e., the way value shifted according to the control input of multiplexer 660.

[0067] Circuit 600 further outputs the control input of multiplexer 660 as shift flag signals 620. Shift flag signals 620 include a respective flag value for each particular LRU data node 610 to indicate whether a shift operation has been (or should be) performed on the way value stored in that particular LRU data node 610. As will be described with reference to FIG. 7, the system can use shift flag signals 620 to perform hazard resolution between two transactions.

[0068] 7 illustrates an exemplary hardware implementation 700 for cache hazard resolution between a current cache transaction and a previous cache transaction. Specifically, the circuit 700 performs hazard resolution when the second phase of the previous cache transaction (for LRU data update) and the first phase of the current cache transaction (for LRU way calculation) are executed in the same clock cycle, as shown in the clk=3 cycle of FIG. 3, where the previous transaction (txn2) is executing the LRU data update and the current transaction (txn3) is executing the LRU way calculation.

[0069] 7, circuit 700 receives several input signals, including (1) eligible way 730, which identifies cache lines eligible for replacement in the set for the current transaction, (2) access way 705 for the previous transaction, (3) shift flag 720, which is output by the previous transaction and specifies shift information for LRU data nodes for LRU data updates in the previous transaction, (4) updated LRU data 740, which specifies way values ​​updated by the previous transaction, (5) unupdated LRU data 710, which specifies way values ​​not updated by the previous transaction, and (6) hazard flag 760, which indicates whether to perform hazard resolution between the previous transaction and the current transaction. Circuit 700 outputs a calculated LRU way 770 for the current transaction.

[0070] Note that input signals 705, 720, and 730 may be available to the current transaction before the LRU data update process is complete for the previous transaction, so data processing using these input signals can begin for the current transaction before the LRU data update process for the previous transaction is complete.

[0071] The circuit 700 includes multiple K×1 multiplexers 702, including MUX0, MUX1, ..., MUX K. Each K×1 multiplexer 702 receives an eligible way 730 as an input. The eligible way 730 can be encoded as a "1" or "0" flag in a K-length vector for the K cache lines in the set, with a "1" flag indicating that the corresponding way is eligible for replacement and a "0" flag indicating that the corresponding way is not eligible for replacement. Each of MUX0, ..., MUX(K-1) also receives a way value from a respective LRU data node 710 as a control input. Note that the way values ​​received by multiplexers MUX0, ..., MUX(K-1) are old way values, i.e., way values ​​that have not been updated by the second phase (T2) of the previous transaction. Multiplexer MUX K receives an access way 705 as a second input. The output of multiplexer 702 identifies an outstanding eligible LRU data node for which the hazard has not been resolved. That is, the output of multiplexer 702 specifies whether the corresponding node stores a stale way value that identifies one of the eligible cache lines. The stale way value refers to a way value that has not been updated by the LRU data update process of a previous transaction.

[0072] The circuit 700 includes a plurality of multiplexers 704 that receive a shift flag signal 720 as a control signal. The data input of each multiplexer 704 is connected to the output of two adjacent K×1 multiplexers 702. Therefore, the multiplexers 704 perform data shifting on the outstanding eligible LRU data nodes based on the shift flag 720 when a hazard exists. The hazard signal 760 indicates that two transactions are performed on the same set, i.e., a hazard exists. Since the shift flag 720 was output by the previous transaction and specifies the shift information of the LRU data node of the previous transaction, the multiplexers 704 perform the same data shifting on the outstanding eligible LRU data nodes as that performed by the previous transaction on the LRU data nodes. Therefore, the output of the multiplexers 704 specifies the LRU data node where the hazard has been resolved.

[0073] The circuit 700 includes a fixed priority arbiter 790 that performs fixed priority arbitration among eligible nodes whose hazards have been resolved, and selects an LRU node 780, i.e., the least recently used node indicated by a node index, from among the eligible nodes whose hazards have been resolved. The circuit 700 can use another K×1 multiplexer 708 to receive the selected LRU node 780 as a control signal to select an LRU way 770 from either the updated LRU data 740 or the unupdated LRU data 710 in response to a hazard signal 760.

[0074] As described above, circuit 700 is configured to perform hazard resolution on eligible LRU data nodes for the current transaction at least partially in parallel with the LRU data update process of the previous transaction. Thus, the time required to determine the access way in the first stage of hazard resolution can be estimated as follows: T1=Max[O(determine hit way), O(select freeway), O(calculate LRU way) + C2 + C (4) Here, C2 is a fixed small time delay for performing additional operations, such as performing data shifts on outstanding eligible nodes.

[0075] Compared to the conventional implementation of the LRU replacement policy, the above implementation saves the following time: ΔT=T conventional -T1>=O(Update LRU data)-C2 (5)

[0076] Because the time to update LRU data increases with cache associativity, the time savings shown in equation (5) can be significant for set-associative caches with many cache lines in each set.

[0077] As shown by the above analysis, by using the cache replacement process described with reference to Figures 2, 3, and 6 and the hazard resolution process described with reference to Figures 5 and 7, the described cache system can overcome certain limitations of conventional LRU replacement implementations and improve time efficiency for performing multiple cache transactions.

[0078] Embodiments of the subject matter and functional operations described herein can be implemented in digital electronic circuitry, tangibly embodied computer software or firmware, or computer hardware, including the structures disclosed herein and their structural equivalents, or one or more combinations thereof. Embodiments of the subject matter described herein can also be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on a tangible, non-transitory storage medium for execution by or to control the operation of a data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random-access or serial-access memory device, or one or more combinations thereof. Alternatively, or in addition, the program instructions can be encoded on an artificially generated transmitted signal, such as a machine-generated electrical, optical, or electromagnetic signal generated to encode information for transmission to a suitable receiver apparatus for execution by a data processing apparatus.

[0079] The term "configured" is used herein in connection with systems and computer program components. To say that one or more computer systems are configured to perform a particular operation or action means that the system has installed thereon software, firmware, hardware, or a combination thereof that, when running, causes the system to perform the operation or action. To say that one or more computer programs are configured to perform a particular operation or action means that the one or more programs contain instructions that, when executed by a data processing apparatus, cause the apparatus to perform the operation or action. Embodiments of the subject matter and functional operations described herein can be implemented in digital electronic circuitry, tangibly embodied computer software or firmware, or computer hardware, including the structures disclosed herein and structural equivalents thereof, or one or more combinations thereof. Embodiments of the subject matter described herein can be implemented as one or more modules of computer program instructions, i.e., as one or more computer programs encoded on a tangible, non-transitory storage medium for execution by or controlling the operation of a data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random-access or serial-access memory device, or one or more combinations thereof. Alternatively or additionally, the program instructions may be encoded in an artificially generated propagated signal, such as a mechanically generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to a receiving device suitable for execution by a data processing device.

[0080] The term "data processing apparatus" refers to data processing hardware and encompasses all kinds of apparatus, devices, and machines for processing data, including, by way of example, a programmable processor, a computer, or multiple processors or computers. An apparatus can also be or further include special-purpose logic circuitry, such as an FPGA (field-programmable gate array) or an ASIC (application-specific integrated circuit). In addition to hardware, an apparatus can optionally include code that creates an execution environment for a computer program, such as code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof.

[0081] A computer program, which may also be referred to or described as a program, software, software application, app, module, software module, script, or code, can be written in any form of programming language, including compiled or interpreted, declarative or procedural, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program, or in multiple cooperating files, e.g., files that store one or more modules, subprograms, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a data communications network.

[0082] The processes and logic flows described herein may be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by special purpose logic circuitry, such as an FPGA or an ASIC, or a combination of special purpose logic circuitry and one or more programmed computers.

[0083] A computer suitable for running a computer program may be based on a general-purpose or special-purpose microprocessor, or both, or on other types of central processing units. Generally, the central processing unit receives instructions and data from a read-only memory, a random-access memory, or both. The essential elements of a computer are a central processing unit for implementing and executing instructions and one or more memory devices for storing instructions and data. The central processing unit and memory may be supplemented by, or incorporated in, special-purpose logic circuitry. Generally, a computer also includes one or more mass storage devices, such as magnetic, magneto-optical, or optical disks, for storing data, or is operatively coupled to receive data from or transmit data to them, or both. However, a computer need not have such devices. Furthermore, a computer may be incorporated in other devices, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), to name just a few.

[0084] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, by way of example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices, magnetic disks such as internal hard disks or removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks.

[0085] While the specification contains many specific implementation details, these should not be construed as limiting the scope of any invention or the scope of patentable subject matter, but rather as descriptions of features that may be unique to particular embodiments of a particular invention. Certain features described herein as separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented in multiple embodiments, either separately or in any suitable subcombination. Furthermore, even if features may be described above as functioning in a particular combination and originally claimed as such, one or more features from a claimed combination may, in some cases, be deleted from the combination, and the claimed combination may be directed to subcombinations or variations of the subcombinations.

[0086] Similarly, while operations are illustrated in the figures and described in the claims in a particular order, this should not be understood as requiring that such operations be performed in the particular order or sequential order shown, or that all of the operations shown be performed, to achieve desirable results. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the program components and systems described may generally be integrated into a single software product or packaged into multiple software products.

[0087] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims may be performed in a different order and still produce desirable results. As an example, the processes depicted in the accompanying figures do not necessarily require the particular order shown or sequential order to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. A cache, a plurality of sets, each set having a plurality of respective ways; and replacement logic configured to implement a two-stage LRU (Least Recently Used) replacement calculation, the two-stage LRU replacement calculation including: a first stage in which the cache calculates a least recently used way for a set; a second step in which the cache updates its LRU data structure with information about the transactionally accessed way; Cache, which causes operations including

2. 2. The cache of claim 1, wherein the cache is configured to perform the two-stage LRU replacement calculation on consecutive clock cycles.

3. The cache of claim 1 , wherein the cache is configured to implement a least recently used (LRU) replacement policy using the calculated least recently used (LRU) way.

4. 2. The cache of claim 1, wherein the replacement logic includes a plurality of nodes that store respective way values ​​for each set, and wherein performing the second stage includes shifting the stored way values ​​between the nodes for the set.

5. 5. The cache of claim 4, wherein performing the second stage includes updating each node with a set that includes data that the node already stores or data shifted from other nodes.

6. Executing the first phase for the current transaction comprises:

6. The cache of claim 5, further comprising: performing hazard resolution for the current transaction in response to a hazard condition resulting from the second phase of a previous transaction being executed on the same particular set.

7. performing the hazard resolution for the current transaction, 7. The cache of claim 6, further comprising: in response to determining that all of the respective ways in the particular set are eligible for replacement, determining the least recently used way for the current transaction based on the least recently used data structure after the least recently used data structure has been updated for the previous transaction.

8. In response to determining that not all of the respective ways in the particular set are eligible for replacement, the replacement logic: determining a plurality of eligible nodes from the plurality of nodes where the hazard has been resolved; selecting an eligible node from the plurality of eligible nodes for which the hazard has been resolved; The cache of claim 6 , configured to determine the least recently used way using the way value stored in the selected eligible node.

9. 9. The cache of claim 8, wherein the replacement logic is configured to begin determining eligible nodes from the plurality of nodes from which the hazard has been resolved before the second phase of the previous transaction completes.

10. Determining a plurality of eligible nodes from the plurality of nodes where the hazard has been resolved includes: generating unresolved node data identifying eligible nodes from the plurality of nodes based on eligible ways and the LRU data structure before the LRU data structure was updated by the second phase of the previous transaction; and shifting the unresolved node data using shift information generated during the second phase of the previous transaction to generate hazard-resolved node data that identifies a plurality of eligible nodes for which the hazard has been resolved.

11. 1. A method for performing an LRU replacement calculation for a cache including multiple sets, each set having multiple respective ways, comprising: Computing a least recently used way for a set during a first phase; updating the LRU data structure with information of the ways that were transactionally accessed during the second phase.

12. 12. The method of claim 11, wherein the first stage and the second stage are performed in consecutive clock cycles.

13. The method of claim 11 , wherein the cache is configured to implement a least recently used (LRU) replacement policy using the calculated least recently used (LRU) way.

14. the method further comprising storing a respective way value for each set of a plurality of nodes; 12. The method of claim 11, wherein performing the second step includes shifting the way values ​​stored between the nodes for the set.

15. 15. The method of claim 14, wherein performing the second stage includes updating each node to a set that includes data that the node already stores or data shifted from other nodes.

16. Executing the first phase for the current transaction comprises:

16. The method of claim 15, further comprising: performing hazard resolution for the current transaction in response to a hazard condition resulting from the second phase of a previous transaction being executed in the same particular set.

17. performing the hazard resolution for the current transaction, 17. The method of claim 16, comprising: in response to determining that all of the respective ways in the particular set are eligible for replacement, determining the least recently used way for the current transaction based on the least recently used data structure after the least recently used data structure has been updated for the previous transaction.

18. performing the hazard resolution for the current transaction in response to determining that not all of the respective ways in the particular set are eligible for replacement; determining a plurality of eligible nodes from the plurality of nodes where the hazard has been resolved; selecting an eligible node from the plurality of eligible nodes from which the hazard has been resolved; and determining the least recently used way using the way value stored in the selected eligible node.

19. 20. The method of claim 18, wherein determining a plurality of eligible nodes from which the hazard has been resolved comprises initiating determining a plurality of eligible nodes from which the hazard has been resolved from the plurality of nodes before the second phase of the previous transaction is completed.

20. Determining a plurality of eligible nodes at which the hazard has been resolved includes: generating unresolved node data identifying eligible nodes from the plurality of nodes based on eligible ways and the LRU data structure before the LRU data structure was updated by the second phase of the previous transaction; shifting the unresolved node data using shift information generated during the second phase of the previous transaction to generate hazard-resolved node data that identifies a plurality of eligible nodes for which the hazard has been resolved; and 20. The method of claim 19, comprising:

Images