Low-latency cache

Abstract

The cache includes a plurality of sets, each set having a plurality of respective ways, and replacement logic configured to execute an LRU replacement policy based on a multi-stage LRU replacement calculation for a transaction. The multi-stages include a first stage in which the cache reads tag data for the transaction and makes a hit determination based on the tag data, a second stage in which the cache reads LRU data for the transaction, and a third stage in which the cache performs the LRU replacement calculation. If the hit determination is a hit, the cache is configured to provide the resulting cache data before completing the third stage.

Description

Background Art

[0001] This specification relates to a system having an integrated circuit device. A cache is a device that stores data fetched from memory or data to be written to memory for one or more different hardware devices within a system. The hardware devices can be different components integrated into a system on a chip (SOC). In this specification, a device that provides read requests and write requests via a cache is referred to as a client device. Some caches respond to memory requests of multiple different client devices integrated in a single system.

[0002] A cache can be used to reduce power consumption by reducing the overall requests to the main memory. Further, as long as the client device can access the required data in the cache, power can be further saved by putting the main memory and the data path to the main memory in a low-power state. Therefore, the amount of cache usage correlates with the overall power consumption, and increasing the amount of cache usage results in a decrease in the overall power consumption. Therefore, devices that rely on battery power, such as mobile computing devices, can extend battery life by increasing the cache usage of the integrated client devices.

[0003] A cache placement policy determines how to place memory blocks in the cache. A least recently used (LRU) replacement policy can be used for a set-associative cache. To execute the LRU replacement policy, the cache system needs to read LRU data that stores the usage recency information of the cache lines in a set. Reading the LRU data can take time and may cause latency in cache transactions.

Summary of the Invention

[0004] This specification describes a caching system that implements a multi-stage low-latency LRU replacement policy for transactions.

[0005] In one particular aspect of this specification, a cache system is provided. The cache system includes a plurality of sets, each having a plurality of respective ways, and substitution logic configured to perform an LRU substitution policy based on a plurality of-stage LRU substitution calculations for a transaction. The plurality of stages include a first stage in which the cache reads the transaction's tag data and makes a hit determination based on that tag data, a second stage in which the cache reads the transaction's LRU data, and a third stage in which the cache performs an LRU substitution calculation, wherein if the hit determination is a hit, the cache is configured to provide the obtained cache data before the third stage is completed.

[0006] In some embodiments of the cache system, if the hit detection is a miss, the cache is configured to issue a read request to memory before the third stage is complete.

[0007] In some embodiments of the cache system, the multi-stage process includes an additional stage that occurs between the second and third stages, during which the cache waits to read LRU data.

[0008] In some embodiments of the cache system, the cache system includes a tag data table separate from the LRU data table, the tag data table is configured to store tag data, and the LRU data table is configured to store LRU data.

[0009] In some embodiments of the cache system, the tag data table and the LRU data table are implemented in separate memory devices.

[0010] In some embodiments of the cache system, the substitution logic is configured to hold attribute data of two previous transactions, including a first transaction and a second transaction, where the first transaction precedes the second transaction, and the second transaction precedes the current transaction, and the substitution logic is further configured to resolve the hazard state for hit determination for the current transaction based on the attribute data.

[0011] In some embodiments of the cache system, in order to resolve a hazard state, the cache system is configured to determine whether the attribute data of a first transaction satisfies a first condition if the hit determination for the current transaction is a hit, the first condition indicating at least that the first transaction is scheduled to allocate an LRU way and that the way information is still pending, and the cache system is further configured to determine the hit determination for the current transaction as pending in response to having determined that the attribute data of the first transaction satisfies the first condition.

[0012] In some embodiments of the cache system, in order to resolve a hazard state, the cache system is configured to determine whether the attribute data of a second transaction satisfies a second condition in response to the determination that the attribute data of a first transaction does not satisfy a first condition, and the second conditions This indicates at least that the second transaction has allocated an LRU way that matches the hit way of the current transaction, and the cache system is further configured to determine the hit determination for the current transaction as a miss in response to a determination that the second transaction satisfies the second condition, and to determine the hit determination for the current transaction as a hit in response to a determination that the second transaction does not satisfy the second condition.

[0013] In some embodiments of the cache system, in order to resolve a hazard condition, the cache system is configured to determine whether the attribute data of a first transaction satisfies a third condition if the hit determination for the current transaction was a miss, the third condition being at least that the first transaction has a tag portion of the same address as the current transaction and that the first transaction is intended to allocate an LRU way, and the cache system is further configured to determine the hit determination for the current transaction as a hit in response to the determination that the attribute data of the first transaction satisfies the third condition.

[0014] In some embodiments of the cache system, in order to resolve a hazard state, the cache system is configured to determine whether the attribute data of a second transaction satisfies a fourth condition in response to the determination that the attribute data of a first transaction does not satisfy a third condition, and the fourth conditions This indicates at least that the second transaction has the same address tag portion as the current transaction and that the second transaction intends to allocate an LRU way, and the cache system is further configured to determine the hit for the current transaction as a hit in response to the determination that the second transaction satisfies the fourth condition, and to determine the hit for the current transaction as a miss in response to the determination that the second transaction does not satisfy the fourth condition.

[0015] Another aspect of this specification provides a method which is performed by the above-described cache system and includes the above-described operation.

[0016] The subject matter described herein can be implemented in specific embodiments to achieve one or more advantages. For example, a cache system may pass hit / miss results to downstream processing without waiting for the reading of LRU data to be completed. This reduces the latency of cache transactions and thus improves the time efficiency of the cache system. Furthermore, in some embodiments, the system stores tag data and LRU data in separate memory devices, thereby increasing design flexibility within cost or resource constraints.

[0017] Details of one or more embodiments of the subject matter described herein are shown in the accompanying drawings and the following description. Other features, aspects, and advantages of the subject matter will become apparent from this specification, the drawings, and the claims. [Brief explanation of the drawing]

[0018] [Figure 1A] Here is an example of a caching system. [Figure 1B] An example of a memory device for storing tag data and LRU data is shown. [Figure 2] This flowchart shows an exemplary process for executing a cache transaction. [Figure 3] This shows an exemplary process for executing a cache transaction. [Figure 4A] The signal waveforms in an exemplary process for executing a cache transaction are shown. [Figure 4B] The following shows signal waveforms in other exemplary processes for executing cache transactions. [Figure 5A] The signal waveforms in an exemplary process for executing two cache transactions are shown. [Figure 5B] The signal waveforms in another exemplary process for executing two cache transactions are shown. [Figure 6]A flowchart showing an exemplary process for performing hazard resolution of a current transaction.

Best Mode for Carrying Out the Invention

[0019] Like reference symbols and designations in the various drawings refer to like elements. The cache placement policy determines how to place memory blocks within the cache. In this specification, the focus is on a set-associative cache placement policy where the cache is divided into a plurality of sets, and each set includes a plurality of cache lines.

[0020] FIG. 1A shows a cache system 100. The cache system 100 may be part of a processing system, such as a system-on-chip (SOC) communicatively connected to a memory device. Specifically, the cache system 100 is a set-associative cache and includes a plurality of sets 110. Each set 110 includes a plurality of respective cache lines 114.

[0021] A cache line, also known as a way, is a unit for transferring data between the cache and other memory devices (e.g., the main memory of the processing system). All cache lines within a set have a fixed size, e.g., 64 bytes. When a read or write is performed at any position within a 64-byte region, the processor reads or writes the entire cache line.

[0022] The cache system 100 further includes a cache transaction controller 120 that manages cache transactions of the cache system 100. As used herein, a cache transaction refers to an access process to the cache system that involves a request for a specific memory block. Exemplary cache transaction processing will be described while referring to FIG. 2. Generally, the cache transaction controller 120 uses index bits derived from the address of the memory block to map the requested memory block to a specific set 110. The cache transaction controller 120 then performs a tag check to determine whether the requested memory block is already located in one of the cache lines 110. The tag check is performed based on tag data 130 that stores tags of all cache lines 114 of the memory device. Specifically, the cache transaction controller 120 compares the tag bits of the address of the memory block with the tags of the cache lines 114 within the mapped set 110. If the tag of the memory block matches any of the cache lines within the mapped set, the tag check returns a "cache hit". If there is no match, the tag check returns a "cache miss".

[0023] In the case of a "cache miss", the cache transaction controller 120 requests the memory block from another memory device such as the main memory of the processing system or the next level cache of the processing system, and places the memory block in a selected cache line 114 within the mapped set 110. If all cache lines 114 within the mapped set 110 are already allocated (i.e., each memory block has been previously placed), the cache transaction controller 120 replaces the block stored in the cache line identified based on the replacement policy with the new data read from the external memory device.

[0024] Specifically, the cache transaction controller 120 uses cache replacement logic 125 to execute a longest unused (LRU) replacement policy, selecting the longest unused cache line to replace (from among the K ways). This process requires tracking the usage recency of each cache line 114 for all other cache lines within a given set 110. Therefore, the system 100 maintains LRU data 140 that specifies the usage recency information of each cache line 114 within each set 110 of the system 100.

[0025] In summary, the cache transaction controller 120 controls the system 100 and executes cache transactions in multiple stages. This includes reading the transaction tag data 130, performing hit / miss determination, reading the transaction LRU data 140, and performing LRU replacement calculations.

[0026] To improve the time efficiency of cache transactions, the system can adjust the start timeline of each operation within the process to minimize latency. An example of an operation timeline is illustrated with reference to Figure 3. Generally, to reduce latency due to reading LRU data 140, if the hit determination is a hit, the cache system is configured to provide the obtained cache data before the execution phase of the LRU replacement calculation is completed. Furthermore, in some embodiments, if the hit determination is a miss, the cache system is configured to issue a read request to memory before the execution phase of the LRU replacement calculation is completed.

[0027] [Table 1]

[0028] In some embodiments, the cache system 100 further includes a data buffer 150 for storing attribute data of previous transactions. The cache transaction controller 120 uses the attribute data to resolve hazards when determining hits / misses for the current transaction. An exemplary process of hazard resolution using attribute data will be described in detail with reference to Figure 6.

[0029] Table 1 shows examples of data fields for tag data 130 and LRU data 140 for a specific cache line. When there are many sets and cache lines in the cache system, tag data 130 and LRU data 140 can occupy a large amount of storage space. In some embodiments, tag data 130 and LRU data 140 are stored separately from each other. For example, as shown in Figure 1B, tag data 130 and LRU data 140 may be stored in two different memory devices, namely a first memory device 151 and a second memory device 152. For example, the first memory device 151 may be a first random access memory (RAM) and the second memory device 152 may be a second RAM.

[0030] Furthermore, as shown in Figure 1B, the tag data 130 may be configured as a first table stored in a first memory device, and the LRU data 140 may be configured as a second table stored in a second memory device 152. The first table can be an N×K table that stores tag information for N sets, with each set having K cache lines. Each cell in the first table stores the tag of a specific cache line within a particular set. Similarly, the second table is also an N×K table that stores recent usage information, with each cell storing the recent usage of a specific cache line within a particular set.

[0031] Storing tag data 130 and LRU data 140 in separate memory devices can improve design flexibility within cost or resource constraints. A well-designed memory system handles significantly more cache hits than cache misses in normal operation. Since cache hits do not require writing to stored tag data, tag writes occur far less frequently than tag reads. In contrast, the frequency of LRU data reads and writes is more balanced. Therefore, to optimize the cost-performance ratio of the system design, tag data can be stored in single-port RAM and LRU data in dual-port RAM.

[0032] Figure 2 shows an exemplary process 200 for executing a cache transaction. For convenience, process 200 is described as being executed by a cache system such as the cache system 100 in Figure 1A.

[0033] Before executing process 200, the system determines the set of cache transactions. The output of process 200 specifies the particular cache line identified by the "access way" in the set to access the memory block specified in the cache transaction request.

[0034] After receiving a cache transaction request specifying a memory block, the system reads the tag data in step 202 and the LRU data in step 204. The system performs a tag check in step 210. Specifically, the system compares the tag bit associated with the address of the memory block with the tag of the cache line in the set.

[0035] In step 210, the system determines whether the tag check is a cache hit or a miss. Specifically, if the tag bit associated with the address of the memory block matches the tag of one of the cache lines in the set, the system determines that the tag check result is a cache hit. For convenience, a cache line with a tag that matches the memory block tag is referred to as a "hitway". If the tag bit associated with the address of the memory block does not match any of the tags of the cache lines in the set, the system determines that the tag check result is a miss.

[0036] If the system determines that the tag check result is a cache hit, the system will use the hit way for the transaction, and therefore assigns the access way to the hit way in step 230.

[0037] If the system determines that the tag check result is a cache miss, this means that the data in the specified memory block is not loaded into any of the cache lines in the set, and the system needs to request the data from the next level of the memory hierarchy and load it into a selected cache line in the set.

[0038] In step 240, the system checks if there are any unused cash lines in the set. For convenience, unused cash lines are referred to as "freeways." If the system determines in step 240 that a freeway is available, in a cache miss scenario, the system uses that freeway to execute the transaction. In other words, in step 260, the system assigns an access way to a freeway.

[0039] If the system determines in step 240 that the freeway is unavailable, this means that all cache lines in the set are in use, and the system must identify which cache lines to replace based on the replacement policy and place the data in the cache lines identified as replacements.

[0040] Since the system implements an LRU substitution policy, in step 270 the system calculates the LRU way. This involves reading the LRU data and determining the LRU way based on the LRU data. In step 280, the system assigns the access way to the LRU way.

[0041] Once the access way is determined (by steps 230, 260, or 280), the system updates the LRU data in step 285 and then writes the LRU data to the second memory device in step 288. The system further updates the tag data in step 265 and writes the tag data to the first memory device in step 268.

[0042] To improve the time efficiency of process 200, the system can adjust the start timeline of each operation within the process to minimize latency. The step of calculating the LRU way (step 270) typically takes longer than the hit or miss determination (step 210). Also, step 270 takes longer than the step of selecting a freeway (step 240). For example, calculating the LRU way may take two clock cycles to complete, while the hit / miss determination and freeway selection may fit within one clock cycle. This is partly because the LRU data is stored in a second memory device, and reading it from there may take more time.

[0043] In some embodiments, to reduce cache transaction latency, once the cache transaction controller determines the hit / miss result in step 210, the system can pass the hit / miss determination to downstream processing without waiting for the LRU data read to complete. For example, if the hit / miss determination is a hit, the system can start the process of retrieving the cached data stored in the hitway without waiting for the LRU data read to complete and provide the retrieved cached data to the transaction. By starting the retrieval of cached data early, the system can make the cached data from the hitway available at an earlier stage, for example, before the LRU substitution calculation is completed. In another example, if the hit / miss determination is a miss, the system issues a read request to the next level of the memory hierarchy without waiting for the LRU data read to complete. Since reading data from the next level of the memory hierarchy can take a considerable amount of time, this strategy can effectively reduce cache transaction latency.

[0044] Figure 3 shows an exemplary timeline for performing operations including tag and memory-related operations 310 and LRU data-related operations 320 for cache transactions, illustrating how these operations fit within a clock cycle. The timeline is divided into four consecutive time segments, T1, T2, T3, and T4. Each time segment may correspond to one or more clock cycles.

[0045] As shown in Figure 3, the system receives an incoming transaction in the first time segment T1 and issues a tag data read request to read the tag data for all cache lines belonging to the set associated with that incoming transaction. Table 2 shows an example of signals processed during the first time segment.

[0046] [Table 2]

[0047] In the second time segment T2, the system issues an LRU data read request to read the LRU data for all cache lines belonging to the set associated with the incoming transaction. Note that the LRU data read operation is delayed by one time segment compared to the tag read operation. This is to account for the fact that it may take more time for the attribute data of the incoming transaction to be passed to the second memory device that stores the LRU data.

[0048] Tag data is available in T2. ​​The system performs tag checks within this segment and then provides the hit / miss determination result as output to the subsequent processing logic. For example, if the hit / miss determination is a hit, the system can start the process of retrieving cached data stored in the hitway without waiting for the LRU data read to complete, and provide the obtained cached data to the transaction. In another example, if the hit / miss determination is a miss, the system can issue a read request to the next level of the memory hierarchy without waiting for the LRU data read to complete. These strategies can effectively reduce the latency of cache transactions. Table 3 shows examples of signals processed during the second time segment.

[0049] After an LRU data read request is issued, the system waits for the LRU data to become available in the third segment T3. Once the read operation makes the LRU data available, the system provides the LRU data for LRU way calculation.

[0050] In the fourth segment T4, the system calculates the LRU way, updates the LRU data, and writes the updated LRU data to the second memory device. The system also updates the tag data and writes the updated tag data to the first memory device. The system also outputs data such as the final hit / miss status and way information. Table 4 shows an example of signals processed during the fourth time segment.

[0051] Figure 4A shows the signal waveforms of the input signal (410A), internal signal (415A) of T1, the output signal (420A) of T2, and the output signal (440A) of T4 in an exemplary process for executing a cache transaction when the hit / miss determination returns a hit. As shown in Figure 4A, the system outputs the hit determination and hit way at T2.

[0052] Figure 4B shows the signal waveforms of the input signal (410B), internal signal (415B) at T1, the output signal (420B) at T2, and the output signal (440B) at T4 when the hit / miss determination returns a miss in an exemplary process for executing a cache transaction. As shown in Figure 4B, at T2, the system outputs data indicating a miss determination and a "waypending" instruction without waiting for the LRU data to become available at T4. At T4, the system retrieves the LRU data and outputs LRU way.

[0053] [Table 3]

[0054] [Table 4]

[0055] As explained with reference to Figures 3, 4A, and 4B, by arranging the operations in cache transactions, the system can reduce latency caused by waiting for LRU data to become available. However, a hazard can occur when determining the hit / miss result of the current transaction in T2 if the previous transaction and the current transaction overlap in time (for example, if the previous transaction is running in T3 or T4), and the overlapping previous transaction is mapped to the same set of items being processed by the current transaction. This is because when the first transaction (e.g., transaction 1) is determined to be a miss and proceeds to T4, the hit / miss indication data for two subsequent transactions mapped to the same set (e.g., transaction 2 which proceeds to T3 and transaction 3 which proceeds to T2) may depend on the access way of transaction 1, which can lead to a hazard.

[0056] For example, as shown in Figure 5A, in the current transaction (e.g., transaction 3) which is determined to be a hit based on the current tag data, the previous transaction (e.g., transaction 1) may have allocated that particular hitway to a different memory block, so the particular "hitway" determined based on the tag data may be a false hit.

[0057] As another example, as shown in Figure 5B, in the current transaction (e.g., transaction 3) which is determined to be a miss based on the current tag data, the correct hit / miss determination may actually be a hit because the previous transaction (e.g., transaction 1) may have misallocated and is waiting for LRU data to allocate the way.

[0058] The reason the above hazard occurs is that the latest tag data from the two previous transactions has not been updated by the time the current transaction performs its hit / miss determination, and the LRU way from the previous transactions is still pending. If such a case is not detected or considered in T2, and only a tag lookup is performed, the hit / miss determination may be incorrect. Note that this hazard can only be caused by the "two previous transactions".

[0059] Figure 6 is a flowchart illustrating an exemplary process for performing hazard resolution for the current transaction. For convenience, process 600 is described as being performed by a caching system, such as caching system 100 in Figure 1A.

[0060] Hazards can occur when the current transaction performs a hit / miss determination in the T2 segment. To ensure reliable results in the hit / miss determination, the system needs to track attribute data from the two previous transactions. Therefore, the system retains attribute data from the two previous transactions (as shown in step 605). The two previous transactions include the first and second transactions, with the first transaction preceding the second. That is, the first transaction is two time segments ahead of the current transaction, and the second transaction is one time segment ahead of the current transaction. When the current transaction proceeds to T2, the first transaction proceeds to T4, and the second transaction proceeds to T3. Table 5 shows an example of attribute data retained for previous transactions.

[0061] In step 610, the system performs a tag check on the current transaction based on the current tag data. Note that the current data may still be updated by any of the previous transactions.

[0062] If the check returns a hit, the system determines in step 615 whether the attribute data of the first transaction satisfies the first condition. The first condition indicates that (1) the first transaction is valid, (2) the set index of the first transaction is the same as that of the current transaction, (3) the tag portion of the transaction address of the first transaction is different from that of the current transaction, and (4) the first transaction is scheduled to allocate an LRU way and the way information is still pending.

[0063] [Table 5]

[0064] If the system determines that the attribute data of the first transaction satisfies the first condition, the system determines the hit / miss status of the current transaction as pending in step 620. The system waits for the completion of the previous transaction to determine the final hit / miss status of the current transaction.

[0065] If the system determines that the attribute data of the first transaction does not satisfy the first condition, the system determines in step 625 whether the attribute data of the second transaction satisfies the second condition. The second condition is that (1) the second transaction is valid, (2) the set index of the second transaction is the same as that of the current transaction, (3) the tag portion of the transaction address of the second transaction is different from that of the current transaction, and (4) the LRU way assigned by the second transaction matches the hit way of the current transaction.

[0066] If the system determines that the attribute data of the second transaction satisfies the second condition, the system determines the hit / miss status of the current transaction as a miss in step 635.

[0067] If the system determines that the attribute data of the second transaction does not satisfy the second condition, the system determines the hit / miss status of the current transaction as a hit in step 630.

[0068] If the judgment returns a miss, in step 640 the system determines whether the attribute data of the first transaction satisfies the third condition. The third condition indicates that (1) the first transaction is valid, (2) the set index of the first transaction is the same as that of the current transaction, (3) the tag portion of the transaction address of the first transaction is the same as that of the current transaction, and (4) the first transaction is scheduled to allocate an LRU way, and the way information is still pending.

[0069] If the system determines that the attribute data of the first transaction satisfies the third condition, in step 645, the system determines the hit / miss status of the current transaction as a hit, but leaves the way information pending. The system waits for the first transaction to determine the LRU way (of T4) and assigns that LRU way as the hit way for the current transaction.

[0070] If the system determines that the attribute data of the first transaction does not satisfy the third condition, the system determines in step 650 whether the attribute data of the second transaction satisfies the fourth condition. The fourth condition is that (1) the second transaction is valid, (2) the set index of the second transaction is the same as that of the current transaction, (3) the tag portion of the transaction address of the second transaction is the same as that of the current transaction, and (4) the second transaction has an LRU way allocated.

[0071] If the system determines that the attribute data of the second transaction satisfies the fourth condition, in step 655, the system determines the hit / miss status of the current transaction as a hit and assigns the hit way of the current transaction as the LRU way of the second transaction.

[0072] If the system determines that the attribute data of the second transaction does not satisfy the fourth condition, the system determines the hit / miss status of the current transaction as a miss in step 660.

[0073] Process 600 takes into account the attributes of the previous two transactions, allowing it to properly resolve hazards when determining the hit / miss status of the current transaction.

[0074] As mentioned above, in order for the system to reduce the latency of cache transactions and improve the time efficiency of the cache system, the cache system can pass the hit / miss results to downstream processing without waiting for the LRU data read to complete. The system can perform processes to resolve any hazards that arise from initiating multiple transactions within consecutive clock cycles.

[0075] The subject matter and functional operating embodiments described herein may be implemented in digital electronic circuits, in tangibly embodied computer software or firmware, in computer hardware, or in one or more combinations thereof, including structures and structural equivalents thereof disclosed herein. Embodiments of the subject matter described herein may be implemented as modules of one or more computer programs, i.e., one or more computer program instructions encoded in a tangible, non-temporary storage medium for execution by or control of the operation of a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage board, a random or serial access memory device, or one or more combinations thereof. Alternatively, or in addition, the program instructions may be encoded into artificially generated propagating signals, for example, machine-generated electrical, optical, or electromagnetic signals generated to encode information generated for transmission to a suitable receiver device for execution by a data processing device.

[0076] The term “configured” as used herein is used in relation to the components of a system and a computer program. A system consisting of one or more computers being configured to perform a particular operation or action means that software, firmware, hardware, or a combination thereof is installed on the system that causes the system to perform that operation or action when it is running. One or more computer programs being configured to perform a particular operation or action means that when the one or more programs are executed by a data processing device, they include instructions that cause that device to perform that operation or action. The subject matter and embodiments of functional operation described herein may be implemented in digital electronic circuits, in tangibly embodied computer software or firmware, in computer hardware, or in one or more combinations thereof, including structures and structural equivalents thereof disclosed herein. Embodiments of subject matter described herein may be implemented as one or more computer programs, i.e., as modules of one or more computer program instructions encoded on a tangible non-temporary storage medium for execution by a data processing device or for controlling the operation of a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage board, a random or serial access memory device, or one or more combinations thereof. Alternatively or additionally, program instructions may be encoded into artificially generated propagating signals, such as mechanically generated electrical, optical, or electromagnetic signals, which are produced to encode information for transmission to a receiving device suitable for execution by a data processing device.

[0077] The term "data processing device" refers to data processing hardware and encompasses all types of devices, machines, and equipment for processing data, including, for example, programmable processors, computers, or multiple processors or computers. A device may also be, or further include, special-purpose logic circuits, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). In addition to hardware, a device may optionally include code that constitutes the execution environment for computer programs, such as processor firmware, protocol stacks, database management systems, operating systems, or one or more combinations thereof.

[0078] Computer programs, as called or described, as programs, software, software applications, apps, modules, software modules, scripts, or code, can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and can be deployed as standalone programs or modules, in any form, including components, subroutines, or other units suitable for use in a computing environment. A program may, but may not, correspond to a file in a file system. A program may be stored in part of a file that holds other programs or data, such as one or more scripts stored in a markup language document, or in a single file dedicated to the program of interest, or in multiple collaborative files (e.g., files that hold one or more modules, subprograms, or parts of code). A computer program may be deployed to run on one computer, or it may be deployed to run on multiple computers located in one place or distributed across multiple locations and interconnected by a data communication network.

[0079] The processes and logic flows described herein can be performed by one or more programmable computers executing one or more computer programs to act on input data and produce outputs. Alternatively, the processes and logic flows can be performed by special-purpose logic circuits, such as FPGAs or ASICs, or by a combination of special-purpose logic circuits and one or more programmed computers.

[0080] A computer suitable for running computer programs may be based on a general-purpose or dedicated microprocessor, or both, or other types of central processing units. Typically, the central processing unit receives instructions and data from read-only memory, random-access memory, or both. The basic elements of a computer are a central processing unit for executing and running instructions, and one or more memory devices for storing instructions and data. The central processing unit and memory may be complemented by or integrated into special-purpose logic circuits. Generally, a computer may also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or be operablely coupled to them to receive data from or transmit data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer may be integrated into other devices, such as mobile phones, personal digital assistants (PDAs), mobile audio or video players, game consoles, Global Positioning System (GPS) receivers, or portable storage devices (such as Universal Serial Bus (USB) flash drives) (these are just a few examples).

[0081] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0082] While this specification includes details of many specific embodiments, these should not be construed as limiting the scope of any invention or claimable content, but rather as descriptions of features that may be specific to a particular embodiment of a particular invention. Certain features described herein as separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described as a single embodiment may also be implemented in multiple embodiments, individually or in any preferred secondary combination. Furthermore, features may be described above as functioning in a particular combination, and even if initially claimed as such, one or more features from the claimed combination may be removed from the combination, and the claimed combination may cover subcombinations or variations of subcombinations.

[0083] Similarly, while operations are shown in the drawings and described in a specific order in the claims, this should not be understood as requiring that such operations be performed in a specific or sequential order shown, or that all shown operations be performed, in order to obtain the desired results. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the program components and systems described can generally be integrated into a single software product or packaged into multiple software products.

[0084] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions described in the claims may be performed in a different order, and this may still yield desirable results. As an example, the process shown in the accompanying figures does not necessarily require that the actions be performed in the specific order or sequence shown to obtain the desired results. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. It's a cache, Multiple sets, each having multiple ways, Includes replacement logic configured to execute an LRU replacement policy based on a multi-stage LRU replacement calculation for a transaction, wherein the multi-stages are: The first step involves the cache reading the tag data of the transaction and performing a hit determination based on the tag data, The second step involves the cache reading the LRU data of the transaction, The third step includes the cache performing an LRU replacement calculation, If the hit determination is a hit, the cache is configured to provide the obtained cache data before the third stage is completed.

2. The cache according to claim 1, wherein if the hit determination is a miss, the cache is configured to issue a read request to memory before the third stage is completed.

3. The cache according to claim 1 or 2, wherein the plurality of stages include an additional stage occurring between the second stage and the third stage, during which the cache waits to read the LRU data.

4. The cache according to claim 1 or 2, wherein the cache includes a tag data table separate from the LRU data table, the tag data table is configured to store tag data, and the LRU data table is configured to store LRUs.

5. The cache according to claim 4, wherein the tag data table and the LRU data table are implemented in separate memory devices.

6. The substitution logic described above is: It is configured to hold attribute data of the two preceding transactions, including the first transaction and the second transaction, wherein the first transaction precedes the second transaction, and the second transaction precedes the current transaction, and the replacement logic further, The cache according to claim 1 or 2, configured to resolve the hazard state at the time of hit determination for the current transaction based on the attribute data.

7. Resolving the aforementioned hazard condition is If the hit determination for the current transaction is a hit, the process includes determining whether the attribute data of the first transaction satisfies a first condition, the first condition being at least indicating that the first transaction intends to allocate an LRU way and that the LRU way is still pending, and resolving the hazard condition further includes The cache according to claim 6, comprising determining the hit determination for the current transaction as pending in response to the determination that the attribute data of the first transaction satisfies the first condition.

8. Resolving the aforementioned hazard condition further involves, In response to the determination that the attribute data of the first transaction does not satisfy the first condition, the process includes determining whether the attribute data of the second transaction satisfies a second condition, the second condition being at least indicating that the second transaction has allocated an LRU way that matches the hit way of the current transaction, and resolving the hazard condition further includes: In response to the determination that the second transaction satisfies the second condition, the hit determination for the current transaction is determined to be a miss, The cache according to claim 7, further comprising determining the hit determination for the current transaction as a hit in response to the determination that the second transaction does not satisfy the second condition.

9. Resolving the aforementioned hazard condition is If the hit determination for the current transaction is determined to be a miss, the process includes determining whether the attribute data of the first transaction satisfies a third condition, the third condition being at least that the first transaction has a tag portion of the same address as the current transaction and that the first transaction is scheduled to allocate an LRU way, and resolving the hazard condition further includes The cache according to claim 6, comprising determining the hit determination for the current transaction as a hit in response to the determination that the attribute data of the first transaction satisfies the third condition.

10. Resolving the aforementioned hazard condition further involves, In response to the determination that the attribute data of the first transaction does not satisfy the third condition, the process includes determining whether the attribute data of the second transaction satisfies the fourth condition, the fourth condition being at least indicating that the second transaction has the same tag portion of the address as the current transaction and that the second transaction intends to allocate an LRU way, and resolving the hazard condition further includes In response to the determination that the second transaction satisfies the fourth condition, the hit determination for the current transaction is determined to be a hit, The cache according to claim 9, further comprising determining the hit determination for the current transaction as a miss in response to the determination that the second transaction does not satisfy the fourth condition.

11. A method for performing an LRU substitution computation on a cache comprising multiple sets, each having multiple ways, wherein the method is: In the first stage, the transaction tag data is read, and a hit determination is made based on the said tag data. In the second stage, the LRU data of the transaction is read, In the third stage, the LRU substitution calculation is performed, A method comprising: if the hit determination is a hit, providing the obtained cached data before the completion of the third stage.

12. The method according to claim 11, further comprising issuing a read request to memory before the third stage is completed if the hit determination is a miss.

13. The method according to claim 11 or 12, further comprising waiting for the LRU data to be read in an additional step occurring between the second and third steps.

14. The method according to claim 11 or 12, further comprising storing tag data in a tag data table and storing LRU data in an LRU data table, wherein the tag data table is separate from the LRU data table.

15. The method according to claim 14, wherein the tag data table and the LRU data table are implemented in separate memory devices.

16. The above method further, The method includes maintaining attribute data of two preceding transactions, including a first transaction and a second transaction, wherein the first transaction precedes the second transaction, and the second transaction precedes the current transaction, and the method further includes, The method according to claim 11 or 12, further comprising resolving the hazard state at the time of hit determination for the current transaction based on the attribute data.

17. Resolving the aforementioned hazard condition is If the hit determination for the current transaction is a hit, the process includes determining whether the attribute data of the first transaction satisfies a first condition, the first condition being at least indicating that the first transaction intends to allocate an LRU way and that the LRU way is still pending, and resolving the hazard condition further includes The method according to claim 16, comprising determining the hit determination for the current transaction as pending in response to the determination that the attribute data of the first transaction satisfies the first condition.

18. Resolving the aforementioned hazard condition further involves, In response to the determination that the attribute data of the first transaction does not satisfy the first condition, the process includes determining whether the attribute data of the second transaction satisfies a second condition, the second condition being at least indicating that the second transaction has allocated an LRU way that matches the hit way of the current transaction, and resolving the hazard condition further includes: In response to the determination that the second transaction satisfies the second condition, the hit determination for the current transaction is determined to be a miss, The method according to claim 17, comprising determining the hit determination for the current transaction as a hit in response to the determination that the second transaction does not satisfy the second condition.

19. Resolving the aforementioned hazard condition is If the hit determination for the current transaction is determined to be a miss, the process includes determining whether the attribute data of the first transaction satisfies a third condition, the third condition being at least that the first transaction has a tag portion of the same address as the current transaction and that the first transaction is scheduled to allocate an LRU way, and resolving the hazard condition further includes The method according to claim 16, comprising determining the hit determination for the current transaction as a hit in response to the determination that the attribute data of the first transaction satisfies the third condition.

20. Resolving the aforementioned hazard condition further involves, In response to the determination that the attribute data of the first transaction does not satisfy the third condition, the process includes determining whether the attribute data of the second transaction satisfies the fourth condition, the fourth condition being at least indicating that the second transaction has the same tag portion of the address as the current transaction and that the second transaction intends to allocate an LRU way, and resolving the hazard condition further includes In response to the determination that the second transaction satisfies the fourth condition, the hit determination for the current transaction is determined to be a hit, The method according to claim 19, comprising determining the hit determination for the current transaction as a miss in response to the determination that the second transaction does not satisfy the fourth condition.

Images