Low latency cache

Abstract

A cache includes: a plurality of groups, wherein each group has a plurality of respective paths; and replacement logic configured to implement the LRU replacement policy based on the LRU replacement calculation in a plurality of stages of the transaction. The plurality of phases includes: a first phase in which the cache reads tag data for the transaction and makes a hit determination based on the tag data; a second stage in which the cache reads the LRU data for the transaction; and a third stage in which the cache performs the LRU replacement computation. If the hit is determined as a hit, the cache is configured to provide the resulting cache data prior to completion of the third stage.

Description

Background Art

[0001] The present description relates to systems having integrated circuit devices.

[0002] A cache is a device that stores data retrieved from or to be written to memory for one or more different hardware devices in a system. The hardware devices can be different components integrated into a system on a chip (SOC). In this specification, the devices that serve read requests and write requests through a cache will be referred to as client devices. Some caches serve memory requests for multiple different client devices integrated into a single system.

[0003] The cache can be used to reduce power consumption by reducing overall requests to main memory. Additionally, as long as the client devices can access the data they need in the cache, further power savings can be achieved by placing the main memory and the data path to the main memory in a low power state. Thus, cache usage is related to overall power consumption, and increasing cache usage produces a reduction in overall power consumption. Thus, devices that rely on battery power (e.g., mobile computing devices) can extend their battery life by increasing cache usage of integrated client devices.

[0004] The cache placement policy determines how memory blocks are placed in the cache. For a set-associative cache, a least recently used (LRU) replacement policy may be used. To implement the LRU replacement policy, the cache system needs to read LRU data, which stores the recency information of cache lines in the set. The reading of LRU data may be time consuming and cause latency in cache transactions. Summary of the invention

[0005] This specification describes a cache system that implements a low-latency LRU replacement strategy during multiple phases of a transaction.

[0006] In one particular aspect of the specification, a cache system is provided. The cache system includes: a plurality of groups, each of which has a plurality of corresponding ways; and replacement logic, the replacement logic being configured to implement an LRU replacement strategy based on an LRU replacement calculation in multiple phases of a transaction. The multiple phases include: a first phase, in which the cache reads tag data for the transaction and performs a hit determination based on the tag data; a second phase, in which the cache reads LRU data for the transaction; a third phase, in which the cache performs an LRU replacement calculation, wherein, when the hit determination is a hit, the cache is configured to provide the resulting cache data before the third phase is completed.

[0007] In some implementations of the cache system, when a hit is determined to be a miss, the cache is configured to issue a read request to the memory before the third stage is completed.

[0008] In some implementations of the cache system, the plurality of phases includes an additional phase occurring between the second phase and the third phase, during which the cache waits for the LRU data to be read.

[0009] In some implementations of the cache system, the cache system includes a tag data table separate from the LRU data table, wherein the tag data table is configured to store tag data, and wherein the LRU data table is configured to store LRU data.

[0010] In some implementations of the cache system, the tag data table and the LRU data table are implemented in separate memory devices.

[0011] In some implementations of the cache system, the replacement logic is configured to: maintain attribute data of two previous transactions including a first transaction and a second transaction, wherein the first transaction precedes the second transaction and the second transaction precedes the current transaction; and resolve a hazard condition for making a hit determination for the current transaction based on the attribute data.

[0012] In some implementations of the cache system, in order to resolve dangerous conditions, the cache system is configured to: when a hit is determined to be a hit for a current transaction, determine whether attribute data of a first transaction satisfies a first condition, the first condition at least indicating that the first transaction will allocate an LRU way and the way information is still pending; in response to determining that the attribute data of the first transaction satisfies the first condition, determine that the hit for the current transaction is determined to be pending.

[0013] In some implementations of the cache system, in order to resolve the hazardous condition, the cache system is configured to: in response to determining that the attribute data of the first transaction does not satisfy the first condition, determine whether the attribute data of the second transaction satisfies the second condition, the second indication at least indicating that the second transaction has been allocated an LRU way that matches the HIT way of the current transaction; in response to determining that the second transaction satisfies the second condition, determine that a hit for the current transaction is determined to be a miss; and in response to determining that the second transaction does not satisfy the second condition, determine that a hit for the current transaction is determined to be a hit.

[0014] In some implementations of the cache system, in order to resolve the hazardous condition, the cache system is configured to: when a hit is determined to be a miss for the current transaction, determine whether attribute data of the first transaction satisfies a third condition, the third condition at least indicating that the first transaction has a tag portion of the same address as the current transaction and the first transaction will allocate an LRU way; and in response to determining that the attribute data of the first transaction satisfies the third condition, determine that the hit for the current transaction is a hit.

[0015] In some implementations of the cache system, to resolve the hazardous condition, the cache system is configured to: in response to determining that the attribute data of the first transaction does not satisfy the third condition, determine whether the attribute data of the second transaction satisfies a fourth condition, the fourth indication at least indicating that the second transaction has a tag portion of the same address as the current transaction and the second transaction will allocate an LRU way; in response to determining that the second transaction satisfies the fourth condition, determine that a hit for the current transaction is determined to be a hit; and in response to determining that the second transaction does not satisfy the fourth condition, determine that the hit for the current transaction is determined to be a miss.

[0016] In another aspect of the present specification, a method is provided, which is executed by the cache system described above and includes the operations described above.

[0017] The subject matter described in this specification can be implemented in a specific implementation to achieve one or more advantages. For example, the cache system passes HIT / MISS results to downstream processing without waiting for the read of LRU data to complete. This reduces the latency of cache transactions and thus improves the time efficiency of the cache system. Further, in some implementations, the system stores tag data and LRU data in separate memory devices, which provides improved design flexibility within cost or resource constraints.

[0018] The details of one or more embodiments of the subject matter described in this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1A An example of a cache system is shown.

[0020] Figure 1B An example of a memory device storing tag data and LRU data is shown.

[0021] Figure 2 is a flow diagram illustrating an example process for performing a cache transaction.

[0022] Figure 3 An example process for performing a cache transaction is shown.

[0023] Figure 4A Signal waveforms during an example process for performing a cache transaction are shown.

[0024] Figure 4B Signal waveforms during another example process for performing a cache transaction are shown.

[0025] Figure 5A Signal waveforms during an example process for performing two cache transactions are shown.

[0026] Figure 5B Signal waveforms during another example process for performing two cache transactions are shown.

[0027] Figure 6 is a flow chart illustrating an example process for performing hazard resolution for a current transaction.

[0028] Like reference numbers and designations in the various drawings indicate like elements. DETAILED DESCRIPTION

[0029] A cache placement policy determines how memory blocks are placed in a cache. This description focuses on a set associative cache placement policy, where the cache is divided into multiple sets and each set includes multiple cache lines.

[0030] Figure 1A A cache system 100 is shown. The cache system 100 may be part of a processing system, such as a system on a chip (SOC) communicatively coupled to a memory device. Specifically, the cache system 100 is a set associative cache and includes a plurality of sets 110. Each set 110 includes a plurality of corresponding cache lines 114.

[0031] A cache line, also called a way, is a unit of data transfer between a cache and another memory device (e.g., the main memory of a processing system). All cache lines in a group have a fixed size, e.g., 64 bytes. When reading or writing to any location in a 64-byte region, the processor reads or writes to the entire cache line.

[0032] The cache system 100 further includes a cache transaction controller 120 that manages cache transactions of the cache system 100. In this specification, a cache transaction refers to a process of accessing the cache system using a request for a specific memory block. Figure 2 An example cache transaction process is described. In general, the cache transaction controller 120 maps the requested memory block to a particular group 110 using index bits derived from the address of the memory block. The cache transaction controller 120 then performs a tag check to determine whether the requested memory block is already placed in one of the cache lines 110. The tag check is performed based on tag data 130, which stores tags for all cache lines 114 of the memory device. Specifically, the cache transaction controller 120 compares the tag bits of the address of the memory block with the tags of the cache lines 114 in the mapped group 110. If the memory block tag matches any of the cache lines in the mapped group, the tag check returns a "cache hit." Otherwise, the tag check returns a "cache miss."

[0033] In the case of a "cache miss," the cache transaction controller 120 requests a memory block from another memory device (such as from the main memory of the processing system or from a next level cache of the processing system) and places the memory block in the selected cache line 114 of the mapped set 110. If all cache lines 114 in the mapped set 110 have been allocated (i.e., the corresponding memory blocks have been previously placed), the cache transaction controller 120 replaces the block stored in the cache line identified by the replacement policy with the new data read from the external memory device.

[0034] Specifically, cache transaction controller 120 uses cache replacement logic 125 to implement a least recently used (LRU) replacement policy that replaces cached data with cached data. The least recently used cache line 114 in a particular set 110 is selected for replacement. This process requires tracking the recency of use of each cache line 114 in a particular set 110 relative to all other cache lines. Therefore, the system 100 maintains LRU data 140 that specifies recency information for each cache line 114 in each set 110 of the system 100.

[0035] In summary, cache transaction controller 120 controls system 100 to perform cache transactions in multiple phases, including reading tag data 130 for the transaction and making HIT / MISS determinations, reading LRU data 140 for the transaction, and performing LRU replacement calculations.

[0036] To improve the time efficiency of cache transactions, the system can arrange the timelines for different operations in the start process to minimize latency. Figure 3 An example of a timeline describing operations. Generally, to reduce latency caused by reading LRU data 140, when a hit is determined to be a hit, the cache system is configured to provide the resulting cache data before the stage of performing LRU replacement calculations is completed. Further, in some implementations, when a hit is determined to be a miss, the cache system is configured to issue a read request to the memory before the stage of performing LRU replacement calculations is completed. Table 1. Cache line tag data and LRU data

[0037] In some implementations, the cache system 100 further includes a data buffer 150 for storing attribute data of previous transactions. The cache transaction controller 120 uses the attribute data to resolve the hazard when determining the HIT / MISS of the current transaction. Figure 6 An example process for using attribute data to resolve hazards is described in detail.

[0038] Table 1 shows an example of data fields of tag data 130 and LRU data 140 for a particular cache line. When there are a large number of groups and cache lines in the cache system, tag data 130 and LRU data 140 may occupy significant storage space. In some implementations, tag data 130 and LRU data 140 are stored separately from each other. For example, Figure 1B As shown in , tag data 130 and LRU data 140 may be stored in two different memory devices, namely, first memory device 151 and second memory device 152. For example, first memory device 151 may be a first random access memory (RAM), and second memory device 152 may be a second RAM.

[0039] Also Figure 1B As shown in , tag data 130 may be organized into a first table stored in a first memory device, and LRU data 140 may be organized into a second table stored in a second memory device 152. The first table may be used to store tag information for N groups. Tables, where each group has K cache lines. Each cell in the first table stores the tag of a specific cache line in a specific group. Similarly, the second table is also used to store recency information A table where each cell stores the recency of a particular cache line in a particular group.

[0040] Storing tag data 130 and LRU data 140 in separate memory devices can provide improved design flexibility within cost or resource constraints. For typical operation, a well-designed memory system handles significantly more cache HITs than cache MISSes. Since cache HITs do not need to be written to the stored tag data, tag writes occur less frequently than tag reads. In contrast, the reading and writing of LRU data is more balanced. Therefore, in order to optimize the cost-performance ratio of the system design, tag data can be stored in a single-port RAM, while LRU data can be stored in a dual-port RAM.

[0041] Figure 2 An example process 200 for performing a cache transaction is shown. For convenience, process 200 will be described as being performed by a cache system such as Figure 1A Cache system 100) is executed.

[0042] Prior to executing process 200, the system has determined a group for a cache transaction. The output of process 200 specifies a particular cache line identified by an "access way" in the group for use in accessing the memory block specified in the cache transaction request.

[0043] After receiving a cache transaction request specifying a memory block, the system reads tag data in step 202 and reads LRU data in step 204. The system performs a tag check in step 210. Specifically, the system compares the tag bits associated with the address of the memory block with the tags of the cache lines in the group.

[0044] In step 210, the system determines whether the tag check results in a cache HIT or a MISS. That is, if the tag bits associated with the address of the memory block match the tag of one of the cache lines in the group, the system determines that the tag check results in a cache HIT. For convenience, the cache line with a tag that matches the memory block tag is referred to as a "HIT way". If the tag bits associated with the address of the memory block do not match any of the tags of the cache lines in the group, the system determines that the tag check result is a MISS.

[0045] If the system determines that the tag check result is a cache HIT, then the system uses the HIT way for the transaction and therefore assigns the access way to the HIT way in step 230 .

[0046] If the system determines that the tag check result is a cache MISS, this means that the data in the specified memory block has not been loaded into any of the cache lines in the group, and the system needs to request the data from the next level in the memory hierarchy and load the data into the selected cache line in the group.

[0047] In step 240, the system checks for cache lines in the group that are not occupied. For convenience, the unoccupied cache lines are referred to as "free ways". If in step 240, the system determines that the free way is available, the system uses the free way to execute the transaction in the cache MISS scenario. That is, in step 260, the system assigns the access way to the free way.

[0048] If in step 240 the system determines that a free way is not available, this means that all cache lines in the group are already occupied and the system needs to identify a cache line for replacement through a replacement policy and place data in the cache line identified for replacement.

[0049] Since the system implements an LRU replacement strategy, the system calculates the LRU way in step 270. This includes reading the LRU data and determining the LRU way based on the LRU data. In step 280, the system assigns the access way to the LRU way.

[0050] Once the access way is determined (by steps 230, 260 or 280), the system updates the LRU data in step 285 and then writes the LRU data to the second memory device in step 288. The system further updates the tag data in step 265 and writes the tag data to the first memory device in step 268.

[0051] In order to improve the time efficiency of process 200, the system can arrange the time lines for different operations in the start process to minimize latency. The step of calculating the LRU way (step 270) usually takes more time than determining HIT or MISS (step 210). Step 270 also takes longer time than selecting an idle way (step 240). As an example, the calculation of the LRU way may take 2 clock cycles to complete, while determining HIT / MISS or selecting an idle way can fit into a single clock cycle. This is partly because the LRU data is stored in the second memory device, and it can take more time to read from the second memory device.

[0052] In some implementations, in order to reduce the latency of cache transactions, once the cache transaction controller determines the HIT / MISS result in step 210, the system can pass the HIT / MISS determination to downstream processing without waiting for the reading of the LRU data to be completed. For example, when the HIT / MISS is determined to be a hit, the system can start the process of retrieving the cache data stored in the HIT way to provide the resulting cache data for the transaction without waiting for the LRU data to be completed. By starting to retrieve the cache data early, the cache data from the HIT way can be used by the system earlier in time (for example, before the LRU replacement calculation is completed). In another example, when the HIT / MISS is determined to be a miss, the system issues a read request to the next level of the memory hierarchy without waiting for the LRU data to be completed. This strategy can effectively reduce the latency of cache transactions because reading data from the next level of the memory hierarchy can take a lot of time.

[0053] Figure 3 An example timeline for performing operations for a cache transaction to illustrate how operations may fit into clock cycles is shown, including tag and memory related operations 310 and LRU data related operations 320. The timeline is divided into four consecutive time periods T1, T2, T3, and T4. Each time period may correspond to one or more clock cycles.

[0054] like Figure 3 As shown in , during a first time period T1, the system receives an incoming transaction and issues a tag data read request for reading tag data of all cache lines belonging to a group associated with the incoming transaction. Table 2 shows an example of signals processed during the first time period. Table 2. Signals processed during the first time period

[0055] During the second time period T2, the system issues an LRU data read request for reading the LRU data of all cache lines belonging to the group associated with the incoming transaction. It should be noted that the LRU data read operation is delayed by a time period compared to the tag read operation. This is to take into account that it may take more time to transfer the attribute data of the incoming transaction to the second memory device storing the LRU data.

[0056] The tag data is available in T2. ​​The system performs a tag check in this segment and then provides the HIT / MISS determination result as an output to the subsequent processing logic. For example, when a HIT / MISS is determined to be a hit, the system can start the process of retrieving the cache data stored in the HIT way to provide the resulting cache data for the transaction without waiting for the LRU data to be completed. In another example, when a HIT / MISS is determined to be a miss, the system can issue a read request to the next level of the memory hierarchy without waiting for the LRU data to be completed. These strategies can effectively reduce the latency of cache transactions. Table 3 shows an example of signals processed during the second time period.

[0057] After the LRU data read request has been issued, the system waits for the LRU data, which will become available in the third segment T3. The system provides the LRU data for LRU way calculation after the LRU data is available from the read operation.

[0058] During the fourth period T4, the system calculates the LRU way, updates the LRU data, and writes the updated LRU data to the second memory device. The system also updates the TAG data and writes the updated TAG data to the first memory device. The system also outputs data such as the final HIT / MISS status and way information. Table 4 shows an example of the signals processed during the fourth time period.

[0059] Figure 4A 1 shows signal waveforms of an input signal (410A) in T1, an internal signal (415A), an output signal (420A) in T2, and an output signal (440A) in T4 during an example process for performing a cache transaction in which a HIT / MISS determination returns a HIT. Figure 4A As shown in FIG. 1 , during T2, the system outputs HIT confirmation and HIT path.

[0060] Figure 4B 1 shows signal waveforms of an input signal (410B) in T1, an internal signal (415B), an output signal (420B) in T2, and an output signal (440B) in T4 during an example process for performing a cache transaction in which a HIT / MISS determination returns a MISS. Figure 4B As shown in , during T2, the system outputs data indicating MISS determination and a "way pending" indication without waiting for LRU data to be available in T4. During T4, the system obtains LRU data and outputs LRU ways. Table 3. Signals processed during the second time period Table 4. Signals processed during the fourth time period

[0061] By reference to the above Figure 3 , Figure 4A and Figure 4B By arranging the operations in the cache transaction as described, the system can reduce the latency caused by waiting for the LRU data to be available. However, when a HIT / MISS result is determined at T2 during the current transaction, if the current transaction overlaps with the previous transaction in time (for example, when the previous transaction is being executed at T3 or T4), and if the overlapping previous transaction is mapped to the same group as the current application, a hazard may occur. This is because when the first transaction (for example, transaction 1) has a MISS determination and it progresses into T4, two subsequent transactions mapped to the same group (for example, transaction 2 that has progressed to T3 and transaction 3 that has progressed to T2) may have their HIT / MISS indication data depending on the access path of transaction 1, which may cause a hazard.

[0062] In one example, if Figure 5A As shown in , for the current transaction (e.g., transaction 3) that determines the HIT result based on the current TAG data, the specific "HIT way" determined based on the TAG data may have been a pseudo-HIT because the previous transaction (e.g., transaction 1) may have allocated the specific HIT way to a different memory block.

[0063] In another example, if Figure 5B As shown in , for the current transaction (e.g., transaction 3) that determines a MISS result based on the current TAG data, the correct HIT / MISS determination may actually be a HIT because the previous transaction (e.g., transaction 1) may have been MISS-allocate and is waiting for LRU data to provide a way for the allocation.

[0064] The reason for the hazardous condition described above is that the latest TAG data of the previous two transactions has not been updated when the current transaction is making HIT / MISS determination, because the LRU way is still pending for the previous transaction. And, if such a situation is not detected or explored in T2 and only the tag lookup is considered, an erroneous conclusion about hit / miss may be made. Here, it should be noted that the hazard can occur only due to "previous two transactions".

[0065] Figure 6 600 is a flow chart illustrating an example process for performing hazard resolution for a current transaction. For convenience, process 600 will be described as being performed by a cache system such as Figure 1A Cache system 100) is executed.

[0066] The hazard may occur during the T2 period of the current transaction when the HIT / MISS determination is made. In order to ensure that the HIT / MISS determination produces reliable results, the system needs to track the attribute data of the two previous transactions. Therefore, the system maintains the attribute data of the previous two transactions (as shown in step 605). The previous two transactions include a first transaction and a second transaction, wherein the first transaction is before the second transaction. That is, the first transaction is two time periods earlier than the current transaction, and the second transaction is one time period earlier than the current transaction. When the current transaction proceeds to T2, the first transaction proceeds to T4, and the second transaction proceeds to T3. Table 5 shows an example of attribute data maintained for previous transactions.

[0067] In step 610, the system performs a tag check for the current transaction based on the current tag data. It should be noted that the current data may still be updated by one of the previous transactions.

[0068] If it is determined that a HIT is returned, then in step 615, the system determines whether the attribute data of the first transaction satisfies a first condition. The first condition indicates that: (1) the first transaction is valid; (2) the first transaction has the same group index as the current transaction; (3) the first transaction has a different tag portion of the transaction address than the current transaction; and (4) the first transaction will allocate an LRU way and the way information is still pending. Table 5 shows the attribute data maintained by the previous transaction

[0069] If the system determines that the attribute data of the first transaction meets the first condition, the system determines the HIT / MISS status of the current transaction as pending in step 620. The system will wait for the previous transaction to complete to determine the final HIT / MISS status of the current transaction.

[0070] If the system determines that the attribute data of the first transaction does not satisfy the first condition, the system determines whether the attribute data of the second transaction satisfies the second condition in step 625. The second condition indicates that: (1) the second transaction is valid; (2) the second transaction has the same group index as the current transaction; (3) the second transaction has a different tag portion of the transaction address than the current transaction; and (4) the second transaction has allocated an LRU way that matches the HIT way of the current transaction.

[0071] If the system determines that the attribute data of the second transaction satisfies the second condition, then in step 635, the system determines that the HIT / MISS status of the current transaction is MISS.

[0072] If the system determines that the attribute data of the second transaction does not satisfy the second condition, then in step 630, the system determines that the HIT / MISS status of the current transaction is HIT.

[0073] If it is determined that MISS is returned, then in step 640, the system determines whether the attribute data of the first transaction satisfies a third condition. The third condition indicates that: (1) the first transaction is valid; (2) the first transaction has the same group index as the current transaction; (3) the first transaction has the same tag portion of the transaction address as the current transaction; (4) the first transaction will allocate an LRU way and the way information is still pending.

[0074] If the system determines that the attribute data of the first transaction satisfies the third condition, then in step 645, the system determines that the HIT / MISS status of the current transaction is HIT, but the way information is pending. The system will wait for the first transaction to determine the LRU way (in T4) and assign the LRU way as the HIT way for the current transaction.

[0075] If the system determines that the attribute data of the first transaction does not satisfy the third condition, then the system determines whether the attribute data of the second transaction satisfies a fourth condition in step 650. The fourth condition indicates that: (1) the second transaction is valid; (2) the second transaction has the same group index as the current transaction; (3) the second transaction has the same tag portion of the transaction address as the current transaction; and (4) the second transaction has an LRU way allocated.

[0076] If the system determines that the attribute data of the second transaction satisfies the fourth condition, then in step 655 , the system determines the HIT / MISS state of the current transaction to be HIT and assigns the HIT way of the current transaction as the LRU way of the second transaction.

[0077] If the system determines that the attribute data of the second transaction does not satisfy the fourth condition, then in step 660, the system determines that the HIT / MISS status of the current transaction is MISS.

[0078] Because process 600 takes into account the properties of the previous two transactions, the hazards in determining the HIT / MISS status of the current transaction can be properly addressed.

[0079] As discussed above, the system reduces the latency of cache transactions and thus improves the time efficiency of the cache system, which can pass HIT / MISS results to downstream processing without waiting for the read of LRU data to complete. The system can implement a process for resolving any hazards caused by starting multiple transactions in consecutive clock cycles.

[0080] Embodiments of the subject matter and functional operations described in this specification may be implemented in digital electronic circuit systems, in tangibly embodied computer software or firmware, in computer hardware (including the structures disclosed in this specification and their structural equivalents), or in a combination of one or more of them. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory storage medium for execution by a data processing device or for controlling the operation of a data processing device. A computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. Alternatively or additionally, program instructions may be encoded on an artificially generated propagation signal (e.g., a machine-generated electrical, optical, or electromagnetic signal) that is generated to encode information for transmission to a suitable receiver device for execution by a data processing device.

[0081] This specification uses the term "configured" in conjunction with system and computer program components. For a system of one or more computers to be configured to perform a specific operation or action, it means that the system has installed thereon software, firmware, hardware, or a combination thereof that causes the system to perform these operations or actions in operation. For one or more computer programs configured to perform a specific operation or action, it means that one or more programs include instructions that cause the device to perform the operation or action when executed by a data processing device. The embodiments of the subject matter and functional operations described in this specification can be implemented in a digital electronic circuit system, in a tangibly embodied computer software or firmware, in computer hardware (including the structures disclosed in this specification and their structural equivalents), or in a combination of one or more of them. The embodiments of the subject matter described in this specification can be implemented as one or more computer programs, that is, one or more modules of computer program instructions encoded on a tangible non-temporary storage medium for data processing equipment to execute or control the operation of the data processing equipment. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. Alternatively or additionally, program instructions may be encoded on an artificially generated propagated signal (e.g., a machine-generated electrical, optical or electromagnetic signal) that is generated to encode the information for transmission to appropriate receiver apparatus for execution by the data processing apparatus.

[0082] The term "data processing equipment" refers to data processing hardware and encompasses all kinds of equipment, devices and machines for processing data, including, for example, a programmable processor, a computer or a plurality of processors or computers. The equipment may also be or further include a dedicated logic circuit system, for example, an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). In addition to hardware, the equipment may also optionally include code that creates an execution environment for a computer program, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0083] A computer program (which may also be referred to or described as a program, software, software application, app, module, software module, script, or code) may be written in any form of programming language including compiled or interpreted languages ​​or declarative or procedural languages, and it may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but need not, correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program may be deployed to execute on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a data communications network.

[0084] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by a dedicated logic circuit system (e.g., an FPGA or ASIC) or by a combination of a dedicated logic circuit system and one or more programmed computers.

[0085] A computer suitable for executing a computer program can be based on a general or special microprocessor or both, or any other kind of central processing unit. Generally, the central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a central processing unit for fulfilling or executing instructions and one or more memory devices for storing instructions and data. The central processing unit and the memory can be supplemented by or incorporated into a special logic circuit system. Generally, the computer will also include one or more large-capacity storage devices for storing data, such as a magnetic disk, a magneto-optical disk, or an optical disk, or operably coupled to receive data from one or more large-capacity storage devices or to transfer data to one or more large-capacity storage devices or both. However, the computer does not have to have such a device. In addition, the computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device, such as a universal serial bus (USB) flash drive, to name a few.

[0086] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices); magnetic disks (e.g., internal hard disks or removable disks); magneto-optical disks; and CD ROM and DVD-ROM disks.

[0087] Although this specification contains many specific implementation details, these details should not be interpreted as limitations on the scope of any invention or the scope that may be claimed, but should be interpreted as descriptions of features that may be peculiar to a particular embodiment of a particular invention. Certain features described in the context of a separate embodiment in this specification may also be implemented in combination in a single embodiment. On the contrary, the various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. In addition, although features may be described above as working in certain combinations and even initially claimed as such, one or more features from the claimed combination may be deleted from the combination in some cases, and the claimed combination may involve a variant of a sub-combination or a sub-combination.

[0088] Similarly, although operations are depicted in the drawings and described in the claims in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in a sequential order, or that all of the operations shown be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0089] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. For example, the actions recited in the claims can be performed in a different order and still achieve the desired results. As an example, the processes depicted in the accompanying drawings do not necessarily require the particular order shown or sequential order to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. A cache comprising: a plurality of groups, wherein each group has a plurality of corresponding ways; as well as A replacement logic configured to implement an LRU replacement policy based on an LRU replacement calculation in multiple phases of a transaction, the multiple phases comprising: a first phase, in which the cache reads tag data for the transaction and makes a hit determination based on the tag data, A second phase, in which the cache reads the LRU data for the transaction, The third stage, in which the cache performs an LRU replacement calculation, Wherein, when the hit is determined to be a hit, the cache is configured to provide the resulting cache data before the third stage is completed.

2. The cache memory of claim 1, wherein: When the hit is determined to be a miss, the cache is configured to issue a read request to memory before the third stage is completed.

3. The cache memory of claim 1, wherein: The plurality of phases includes an additional phase occurring between the second phase and the third phase, during which the cache waits for the LRU data to be read.

4. The cache memory of claim 1, wherein: The cache includes a tag data table separate from an LRU data table, wherein the tag data table is configured to store tag data, and wherein the LRU data table is configured to store LRU data.

5. The cache of claim 4, wherein: The tag data table and the LRU data table are implemented in separate memory devices.

6. The cache memory of claim 1, wherein: The replacement logic is configured to: maintaining attribute data of two previous transactions including a first transaction and a second transaction, wherein the first transaction precedes the second transaction and the second transaction precedes the current transaction; and A risk condition for making the hit determination for the current transaction is resolved based on the attribute data.

7. The cache memory of claim 6, wherein: Addressing the hazardous conditions includes: When the hit determination is a hit for the current transaction, determining whether the attribute data of the first transaction satisfies a first condition, the first condition at least indicating that the first transaction will allocate an LRU way and the way information is still pending; and In response to determining that the attribute data of the first transaction satisfies the first condition, the hit determination for the current transaction is determined to be pending.

8. The cache of claim 7, wherein: Addressing the hazardous conditions further includes: In response to determining that the attribute data of the first transaction does not satisfy the first condition, determining whether the attribute data of the second transaction satisfies a second condition, the second indication at least indicating that the second transaction has been allocated an LRU way that matches the HIT way of the current transaction; In response to determining that the second transaction satisfies the second condition, determining that the hit for the current transaction is determined to be a miss; and In response to determining that the second transaction does not satisfy the second condition, the hit for the current transaction is determined to be a hit.

9. The cache memory of claim 6, wherein: Addressing the hazardous conditions includes: When the hit determination is a miss for the current transaction, determining whether the attribute data of the first transaction satisfies a third condition, the third condition at least indicating that the first transaction has the same tag portion of the address as the current transaction and the first transaction is to be allocated an LRU way; and In response to determining that the attribute data of the first transaction satisfies the third condition, the hit determination for the current transaction is determined to be a hit.

10. The cache of claim 9, wherein: Addressing the hazardous conditions further includes: In response to determining that the attribute data of the first transaction does not satisfy the third condition, determining whether the attribute data of the second transaction satisfies a fourth condition, the fourth indication at least indicating that the second transaction has the same tag portion of the address as the current transaction and the second transaction is to be allocated an LRU way; In response to determining that the second transaction satisfies the fourth condition, determining that the hit for the current transaction is a hit; and In response to determining that the second transaction does not satisfy the fourth condition, determining the hit for the current transaction is determined to be a miss.

11. A method for performing an LRU replacement calculation for a cache comprising a plurality of sets, wherein each set has a plurality of corresponding ways, the method comprising: reading tag data for the transaction during a first phase and making a hit determination based on the tag data; reading LRU data for the transaction during a second phase; During the third phase, the LRU replacement calculation is performed; as well as When the hit is determined to be a hit, the resulting cache data is provided before the third stage is completed.

12. The method of claim 11, further comprising: When the hit is determined to be a miss, a read request is issued to the memory before the third stage is completed.

13. The method of claim 11, further comprising: Waiting for the LRU data to be read during an additional phase occurring between the second phase and the third phase.

14. The method of claim 11, further comprising: The tag data is stored in a tag data table and the LRU data is stored in an LRU data table, wherein the tag data table is separate from the LRU data table.

15. The method of claim 14, wherein: The tag data table and the LRU data table are implemented in separate memory devices.

16. The method of claim 11, further comprising: maintaining attribute data of two previous transactions including a first transaction and a second transaction, wherein the first transaction precedes the second transaction and the second transaction precedes the current transaction; and A risk condition for making the hit determination for the current transaction is resolved based on the attribute data.

17. The method of claim 16, wherein: Addressing the hazardous conditions includes: When the hit determination is a hit for the current transaction, determining whether the attribute data of the first transaction satisfies a first condition, the first condition at least indicating that the first transaction will allocate an LRU way and the way information is still pending; and In response to determining that the attribute data of the first transaction satisfies the first condition, the hit determination for the current transaction is determined to be pending.

18. The method of claim 17, wherein: Addressing the hazardous conditions further includes: In response to determining that the attribute data of the first transaction does not satisfy the first condition, determining whether the attribute data of the second transaction satisfies a second condition, the second indication at least indicating that the second transaction has been allocated an LRU way that matches the HIT way of the current transaction; In response to determining that the second transaction satisfies the second condition, determining that the hit for the current transaction is determined to be a miss; and In response to determining that the second transaction does not satisfy the second condition, the hit for the current transaction is determined to be a hit.

19. The method of claim 16, wherein: Addressing the hazardous conditions includes: When the hit determination is a miss for the current transaction, determining whether the attribute data of the first transaction satisfies a third condition, the third condition at least indicating that the first transaction has the same tag portion of the address as the current transaction and the first transaction is to be allocated an LRU way; and In response to determining that the attribute data of the first transaction satisfies the third condition, the hit determination for the current transaction is determined to be a hit.

20. The method of claim 19, wherein: Addressing the hazardous conditions further includes: In response to determining that the attribute data of the first transaction does not satisfy the third condition, determining whether the attribute data of the second transaction satisfies a fourth condition, the fourth indication at least indicating that the second transaction has the same tag portion of the address as the current transaction and the second transaction is to be allocated an LRU way; In response to determining that the second transaction satisfies the fourth condition, determining that the hit for the current transaction is a hit; and In response to determining that the second transaction does not satisfy the fourth condition, determining the hit for the current transaction is determined to be a miss.

Images