Caching guidelines informing about criticality

Abstract

Furnishings, comprehensive: one or more processors (10A - 10N) configured to issue memory requests for accessing a memory system (18); and a cache switching logic (32) configured to cache data from the storage system (18) for access by the one or more processors (10A - 10N), wherein: the cache switching logic (32) includes a control switching logic (34) that is configured to: Assigning criticality values ​​to cache rows, where a given criticality value indicates that the corresponding cache row is not critical or has one of several criticality levels; and Adjusting the criticality value of a cache row in response to a criticality event; where the cache switching logic (32) to select a sacrificial cache line for replacement is configured to: Masking cache lines that have one or more higher criticality levels; and Selecting the victim cache row from unmasked cache rows based on access frequency data managed separately from criticality values ​​by the cache switching logic (32).

Description

BACKGROUND Technical area

[0001] The embodiments described herein relate to caches in computer systems and, in particular, to caching policies. Description of the state of the art

[0002] Caches have long been used in digital systems to effectively reduce memory latency by storing a copy of data accessed by a processor, coprocessor, or other digital device in a cache memory located locally on the device. The cache memory can be smaller than the main memory system and can be optimized for low latency (whereas the main memory system is often optimized for memory density, which comes at the cost of latency). Accordingly, the cache memory itself can reduce latency. Additionally, the cache memory can reside locally on the device, thus reducing latency because there is no transport delay to the memory controller / main memory system and back to the device. Furthermore, the cache can be private to the device or a small number of devices (e.g.,processor / coprocessor cluster), and thus the competition for bandwidth to the cache compared to main memory can be reduced.

[0003] While caches effectively reduce memory latency, they represent limited storage and are therefore prone to misses (which cause data to be filled from memory into the cache to preserve the data, in addition to serving the data to the requesting device if the miss is a read request, or performing the update if the miss is a write request). Memory in the cache (e.g., a cache row or block) is allocated to the fill. This allocation may necessitate replacing other data in the cache (also known as exposing a cache row from the cache). A variety of replacement policies exist for selecting the cache row to be exposed, based on the cache geometry.For example, set-associative caches have memory arranged as a two-dimensional array of cache rows: A "row" is selected based on a subset of the memory address of the cache row (called a set), and the row includes a multitude of cache rows that are the "columns" of the array (called paths). When a cache miss is detected and a fill is initiated, the fill is assigned to one of the paths. A popular replacement policy for set-associative caches is the least recently used (LRU) policy. With LRU, accesses to the cache rows in a set are tracked from the most recently used (MRU) to the least recently used (LRU).Typically, when a cache row is accessed, its MRU is updated, and the cache rows between the former rank of the cache row and the previous MRU are adjusted. The LRU cache row can be selected for replacement if a cache miss occurs.

[0004] US 2011 / 0 145 506 A1 discloses a cache memory whose cache lines store a weight that indicates the importance of the data based on coherence state and frequency of use.

[0005] US 2021 / 0 390 053 A1 discloses a host-assisted prefetcher in which the host provides the prefetch configuration to reduce the resource overhead on the storage device. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The following detailed description refers to the accompanying drawings, which will now be briefly described. Fig. Figure 1 is a block diagram of an embodiment of a section of a system. Fig. Figure 2 is a flowchart illustrating a criticality determination for an embodiment. Fig. 3 is a table that shows an LRU insertion and update in one embodiment of a in Fig. 1 illustrates the Last-Level-Cache (LLC) shown. Fig. 4 is a flowchart illustrating one implementation of victim selection in the LLC. Fig. Figure 5 is a flowchart illustrating a criticality determination for a different embodiment. Fig. 6 is a flowchart illustrating an LRU insertion in the LLC for one embodiment. Fig. Figure 7 is a flowchart illustrating an LRU upgrade in LLC for one implementation. Fig. Figure 8 is a flowchart illustrating a sacrifice selection process for an embodiment. Fig. Figure 9 is a flowchart illustrating a sacrifice selection for a different embodiment. Fig. Figure 10 is a flowchart illustrating an emission acceleration for cache rows marked as critical. Fig. Figure 11 is a flowchart illustrating an LRU insertion for cache rows in the memory cache for one embodiment. Fig. Figure 12 is a block diagram of an embodiment of a system-on-a-chip (SoC). Fig. Figure 13 is a block diagram of different embodiments of a computer system. Fig. Figure 14 is a block diagram of an embodiment of a computer-accessible storage medium.

[0007] While embodiments described in this disclosure may be subject to various modifications and alternative forms, specific embodiments are shown by way of example in the drawings and are described in detail herein. It is understood, however, that the drawings and the detailed description thereto are not intended to limit the embodiments of the disclosed particular form, but rather that, on the contrary, they are intended to cover all modifications, equivalents, and alternatives that are within the nature and scope of protection of the accompanying patent claims. The headings used herein serve only organizational purposes and are not intended to limit the scope of the description. DETAILED DESCRIPTION OF EXECUTION FORMS

[0008] The invention is set out in the attached claims.

[0009] While the LRU replacement policy often provides good performance (e.g., cache hit rates remain high, thus effectively reducing memory latency), there are cases where performance can be limited. For example, if competition for cache rows is high, resulting in frequent release, some cache rows may be released that, when accessed again, cause a greater performance degradation for the requesting device than other cache rows. For instance, if several operations on the requesting device depend directly or indirectly on data in a cache row, the requesting device may stall while waiting for that data. Other cache rows with fewer dependencies may be less performance-critical. The LRU policy has no way of reflecting these differences in the criticality of cache rows.

[0010] In one embodiment, a system comprising one or more processors and a cache coupled to the one or more processors can categorize cache rows according to one or more criticality levels based on one or more criteria measured at the time the cache rows are filled into the cache. The criteria can be selected to attempt to identify the cache rows that, if they represent a cache miss, will have a greater impact on processor performance than other cache rows. Each cache row can have a criticality value indicating its criticality level. For example, the critical value can indicate a non-critical state or a critical state. In one embodiment, the critical state can also have multiple criticality levels, as described in more detail below.In another embodiment, the critical status can be a single level that indicates critical, as opposed to the non-critical status.

[0011] The cache can implement a replacement policy that uses the criticality values ​​of the cache rows as a factor. For example, an LRU policy can be used, but the policy can be modified to account for the criticality of different cache rows. Cache rows with a criticality value indicating a critical status ("critical cache rows") can be inserted into the LRU replacement data at the MRU position, while cache rows with criticality values ​​indicating a non-critical status ("non-critical cache rows") can be inserted at lower positions in the data (e.g., closer to the LRU position). In one embodiment, criticality values ​​can also influence the updating of the LRU replacement data. While LRU is used as an example replacement policy, other embodiments can implement different replacement policies.For example, a variety of pseudo-LRU policies can be used that approximate the LRU operation by making simplifications to make the policy easier to implement, especially in large record-associative caches. Random replacement policies can also be used, and criticality can be used to reduce the probability of selecting critical rows. Least Used policies can be used, and critical rows can be selectively retained in a manner similar to that described below for LRU. Last In, First Out or First In, First Out policies can be used, and critical cache rows can be at least partially exempted from LIFO or FIFO replacement. Each of these policies can be modified to account for criticality.

[0012] In one embodiment, the system can include one or more additional cache levels between the aforementioned cache and system memory. For example, a memory cache can be used, implemented at the memory controller that manages system memory. The criticality values ​​of cache lines can be exchanged between caches when the cache lines are exposed and re-accessed, with the criticality values ​​being retained while the cache lines remain cached in the cache hierarchy. Once the cache line is removed from the cache hierarchy (and thus the data exists only in system memory), the criticality value can be lost.

[0013] Fig. Figure 1 is a block diagram of an embodiment of a system that includes a plurality of processors 10A-10N, a coprocessor 12, a last-level cache (LLC) 14, a memory controller 16, and a memory 18. The processors 10A-10N and the coprocessor 12 are coupled to the LLC 14, which is coupled to the memory controller 16, which is further coupled to the memory 18. Processor 10N is illustrated in more detail, and other processors, such as processor 10A, may be similar. Processor 10N may include an instruction cache (ICache) 20, an instruction cache miss queue (IC miss queue) 22, an execution core 24 including a load queue (LDQ) 26, a data cache (DCache) 28, and a memory management unit (MMU) 30. The LLC 14 can include a cache 32, a criticality control circuit 34 and a memory cache insert lookup table (MCache insert LUT) 36.The memory cache 16 can include an insertion control circuit and LUT 38, an MCache 40 and a monitoring circuit 42.

[0014] ICache 20 can store instructions that are retrieved by processor 10N for execution by execution kernel 24. If a retrieval from ICache 20 fails, the retrieval of the instruction cache line can be placed in IC miss queue 22 and transferred to LLC 14 as a refill request for ICache 20. Instructions executed by execution kernel 24 can include load instructions (or simply, loads). The loads can attempt to read data from DCache 28 and, if a load in DCache 28 fails, be transferred to LLC 14 as a refill request for DCache 28. The loads transferred to LLC 14 can remain in LDQ 26 to wait for data.

[0015] The MMU 30 can provide address translations for instruction fetch addresses and load / store addresses, including translation buffers (TLBs) that can reside locally in the ICache 20 and the execution kernel 24. The MMU 30 can optionally include one or more Level 2 TLBs (L2 TLBs) and table traverse switching logic to perform the translation table reads to obtain a translation for an address that causes misses in the TLBs. The MMU 30 can transfer the table traverse reads to the LLC 14. In one embodiment, the MMU 30 can access the DCache 28 for a potential cache hit on the table traverse reads before transferring them to the LLC 14, and can choose not to transfer the reads to the LLC 14 if they encounter the DCache 28.In other embodiments, the page table data is not cached in the DCache 28 and the MMU 30 can transfer table traverse read operations to the LLC 14.

[0016] LLC 14 includes Cache 32, which can have any capacity and configuration. Memory requests from processors 10A-10N and coprocessor 12 can be checked for a match in Cache 32, and if a match is found, the data can be returned as a fill to ICache 20, DCache 28, or MMU 30. If the memory request results in a miss in Cache 32, LLC 14 can transfer a memory request to Memory Controller 16 and, in response to Memory Controller 16 returning a fill to LLC 14, can return the fill to the requesting processor 10A-10N or coprocessor 12. LLC 14 can also fill the data into Cache 32 in the event of a miss.In general, “data” is used herein in a general sense, so that it refers both to instructions requested by the processors 10A-10N for execution and to data read / written by the processors as a result of the execution of the instructions (e.g. operand data and result data), especially when it comes to cache rows of data.

[0017] Additionally, at the time of filling the cache line to processor 10A-10N / coprocessor 12, LLC 14 can assign a criticality value. Criticality control circuit 34 can determine the criticality value and update the cache 32 with it. For example, the cache tags in cache 32 can include a field for the criticality value. The critical value can indicate a non-critical state or a critical state. As mentioned above, in some embodiments there can be more than one level of critical state. Criticality control circuit 34 can also determine the level of the critical state.

[0018] The criticality control circuit 34 can consider a variety of factors when assigning criticality values ​​to cache rows. For example, the criticality control circuit 34 is coupled to the MMU 30, the IC miss queue 22, and the LDQ 26. In particular, fills that are for table traverse requests can be categorized as critical. A TLB miss will likely affect additional instruction fetches or load / memory requests because a translation involves a fairly large amount of data, and code sequences tend to access data that is close to other recently fetched data. For example, a page can be 4 kilobytes, 16 kilobytes, or even larger, such as 1 megabyte or 2 megabytes. Any page size can be used. Additionally, a load that is at the head of the LDQ 26 when the fill for the load occurs might be the oldest pending load in the processor 10N.Thus, it is likely that the load will block the exit of other completed instructions, or that a number of instructions will be blocked due to dependencies on the load data (either directly or indirectly). Fills for loads located at the head of LDQ 25 can be assigned a critical status. Similarly, if a fill exists for an instruction fetch request and it is the oldest fetch request in IC miss queue 22 (e.g., located at the head of IC miss queue 22), it is likely that the retrieval of instructions will be blocked by waiting for instructions. Such instruction fetch requests can be assigned a critical status. Other embodiments may, as desired, include additional factors within a given processor 10A-10N, or subsets of the aforementioned factors and other factors.In one embodiment, a critical status can also be assigned to the requirements of the coprocessor 12. For example, one embodiment of the coprocessor 12 might not include a cache, and thus the LLC 14 is the first intermediate storage stage available to the coprocessor 12. Cache rows that have not been assigned a critical status can be assigned a non-critical status.

[0019] In one embodiment, the criticality values ​​assigned to cache rows can be retained while the cache rows remain valid in the cache hierarchy. The criticality value is assigned by the criticality control circuit 34 and then propagated with the cache row when it is released from cache 32 and transferred to the memory controller 16, where it can be temporarily stored in the MCache 40. If the released cache row is placed in the MCache 40 after being released from cache 32, the criticality value can be retained. If the released cache row is not placed in the MCache 40 after being released from cache 32, the memory controller 16 can drop the criticality value and write the data to memory 18. A variety of factors can affect whether or not an released cache row is temporarily stored in the MCache 40.MCache 40 is shared with other system components, and it may have quotas for how much data from a given component can be cached. If LLC 14 exceeds its quota, the exposed cache row might not be cached. Alternatively, the exposed cache row might be cached, and another LLC cache row cached in MCache 40 could be exposed.

[0020] Subsequently, if a cache row previously cached by LLC 14 is accessed again by LLC 14, MCache 40 can provide the cache row to Cache 32 as a fill, and the criticality value previously assigned to the cache row can also be provided. Criticality control circuit 34 can assign the previous criticality value provided to the cache row by MCache 40 unless other factors from processor 10A-10N, which initiated the re-access to the cache row, indicate an update to a critical state or a higher level of critical state. For example, a non-critical cache row from MCache 40 with a non-critical state can be filled into LLC 14 unless it is assigned a critical state at the time of the fill for re-access (e.g.,(if the fill is for a load at the head of LDQ 26, an instruction call at the head of IC miss queue 22, or an MMU table pass request). A critical cache row from MCache 40 can be filled as critical. In embodiments implementing multiple criticality status levels, a critical cache row from MCache 40, which is currently also identified as critical by the factors mentioned above (head of LDQ 26, head of IC miss queue 22, or MMU request), can be assigned a higher critical status level by the criticality control circuit 34.

[0021] In one embodiment, cache rows emitted from LLC 14 can be temporarily stored in MCache 40 and inserted into the replacement data of the affected set in MCache 40 at a selected position. If the emitted cache row is a critical cache row, it can be inserted at the MRU position. If the emitted cache row is a non-critical cache row, it can be inserted at a position lower than the MRU (closer to the LRU). In one embodiment, the insertion point for non-critical cache rows can be dynamic. For example, the insertion point can be based on the extent of cache capacity in MCache 40 occupied by cache rows from LLC 14. The memory controller 16 can include a monitoring circuit 42 that monitors the capacity of the MCache 40 assigned to the CPU and provides the information (“CPU capacity”) to the criticality control circuit 34.The criticality control circuit 34 can use the CPU capacity value as an index to the MCache insertion LUT 36 and can read an insertion hint from the indexed entry when an exposed cache row is transferred to the memory controller 16. The insertion hint can be used as an index to a LUT 38 in the memory controller 16, and the associated insertion control logic can potentially adjust the insertion point (e.g., if a section of the cache is disabled, the insertion point should be within the currently used LRU positions). The MCache 40 can insert the exposed cache block at the insertion point.

[0022] Accordingly, in this embodiment, cooperative lookup tables can be used to determine the insertion point for exposed cache rows in the MCache 40 for non-critical cache rows. The LUTs can be programmable, allowing the software to optimize performance as desired.

[0023] The CPU capacity value can be measured in any desired manner. In one embodiment, the CPU capacity can indicate the average number of MCache paths occupied by cache rows from LLC 14. In another embodiment, an approximate percentage of the cache capacity can be provided.

[0024] As mentioned previously, Cache 32 can have a field (e.g., in the cache tag) for the criticality value. Similarly, MCache 40 can include a field in the cache tag for the criticality value. In another embodiment, MCache 40 can have a record identifier (DSID) for each cache row, which identifies related cache rows according to one or more criteria. Generally, cache blocks with the same DSID can originate from the same source component (e.g., LLC 14 or another component of the system, such as one in Fig. (1 peripheral component not shown). The DSID can be stored in a field within the tag. The DSID can be used to distinguish between non-critical and critical cache rows (e.g., by using one DSID for non-critical cache rows and another DSID for critical cache rows, or multiple DSIDs for different levels of critical status in embodiments that use multiple levels of critical status). The MCache 40 can decode the DSID to determine the criticality value to be transmitted to the LLC 14 when deploying a fill.

[0025] In one embodiment, the 10A-10N processors can serve as the central processing unit (CPU) of the system. The system's CPU includes the one or more processors that execute the system's main control software, such as an operating system. Generally, software executed by the CPU during operation can control the other system components to achieve the desired system functionality. The 10A-10N processors can also execute other software, such as application programs. These application programs can provide user functionality and can access the operating system for lower-level device control, scheduling, memory management, and other functions. Accordingly, the 10A-10N processors can also be referred to as application processors.

[0026] In general, a processor can include any switching logic and / or microcode configured to execute instructions defined in an instruction set architecture implemented by the processor. Processors can include processor cores implemented with other components as a system-on-a-chip (SoC) on an integrated circuit or at other integration levels. Processors can further include discrete microprocessors, as well as processor cores and / or microprocessors integrated in multi-chip module implementations; processors implemented as multiple integrated circuits; and so on.

[0027] In one embodiment, the coprocessor 12 can be configured to accelerate certain operations. For example, one embodiment is considered in which a coprocessor performs large-scale matrix and vector manipulations (multiple operations per instruction). The coprocessor 12 can receive instructions transmitted by the processors 10A-10N. That is, the instructions executed by the coprocessor 12 ("coprocessor instructions") and the instructions executed by the processors 10A-10N ("processor instructions") can be part of the same instruction set architecture and be mixed in a code sequence fetched by the processor. The processor 10A-10N can decode the instructions, identify the coprocessor instructions for transmission to the coprocessor 12, and execute the processor instructions.The coprocessor 12 can receive coprocessor instructions from the processor 10A-10N, decode the coprocessor instructions, and execute the coprocessor instructions. The coprocessor instructions can include load / store instructions to read memory data for operands and write result data to memory (in one embodiment, both can be performed in the LLC 14).

[0028] It is noted that the number and type of different components of the system in Fig. 1. The configuration may vary from embodiment to embodiment. For example, any number of processors 10A-10N may be present. More than one coprocessor 12 may be present, and if multiple coprocessors are included, there may be multiple instances of the same coprocessor and / or different types of coprocessors. More than one memory controller 16 may be present, and if multiple memory controllers are included, the memory space may be distributed across the memory controllers.

[0029] It is noted that various instructions, memory requests, etc., are referred to above as younger or older than other instructions, requests, etc. A given operation may be younger than another operation if the given operation is derived from an instruction that occurs chronologically after the instruction from which the other operation is derived, in program order. Similarly, a given operation is older than another operation if the given operation is derived from an instruction that occurs before the instruction from which the other operation is derived, in program order.

[0030] The Fig. 2, Fig. 3 to Fig. Figure 4 illustrates an embodiment in which criticality values ​​are either critical or non-critical. Fig. 5, Fig. 6, Fig. 7, Fig. 8 to Fig. Figure 9 illustrates an embodiment in which the critical status has more than one criticality level. Fig. Figure 10 illustrates a mechanism for accelerating the removal of critical cache lines from LLC 14 for an embodiment that can apply to both types of criticality values. Fig. 11 is a flowchart illustrating the victim selection from LLC 14 based on the acceleration mechanism of Fig. 10 illustrated.

[0031] With reference to Fig. Figure 2 shows a flowchart illustrating one embodiment of the criticality control circuit 34 for assigning a criticality value to a cache row being filled in the LLC 14. Although the blocks are shown in a specific order for ease of understanding, other sequences can be used. Blocks can be executed in parallel using combinational logic in the criticality control circuit 34. Blocks, combinations of blocks, and / or the flowchart as a whole can be executed over multiple clock cycles. The criticality control circuit 34 can be configured to perform the following actions in Figure 2: Fig. 2. To implement the operation shown.

[0032] If the fill is a cache row for an MMU table traverse request (decision block 50, "yes" branch), criticality control circuit 34 can assign a critical status to the criticality value associated with the cache row (block 52). If the fill is a cache row for a load operation located at the head of LDQ 26 (decision block 54, "yes" branch), criticality control circuit 34 can assign a critical status to the criticality value associated with the cache row (block 52). If the fill is a cache line for an instruction cache miss located at the head of IC miss queue 22 (decision block 56, "yes" branch), the criticality control circuit 34 can assign a critical status to the criticality value associated with the cache line (block 52).If the fill is a cache row that has a critical status in the MCache (decision block 58, "yes" branch), the criticality control circuit 34 can assign a critical status to the criticality value associated with the cache row (block 52). If none of the above criteria apply (decision blocks 50, 54, 56, 58, and 60, "no" branches), the criticality control circuit 34 can assign a non-critical status to the criticality value associated with the cache row. In one embodiment, the coprocessor requests from coprocessor 12 can also be assigned a critical status. In another embodiment, the coprocessor requests can be assigned a non-critical status.

[0033] Fig. Figure 3 is a Table 62 illustrating the operation of an embodiment of the criticality control circuit 34 for updating the substitute data for a set based on a fill of a cache row into the cache 32 (insertion section 64) and based on a cache hit for a processor request from a processor 10A-10N (update section 66). The update of the substitute data can be based on the request type, the previous state of the cache block, and the criticality value. The LRU column of the table indicates the position in the LRU ranking (from MRU to LRU) of the cache row being filled (in the insertion section 64) or the cache row being hit by a request (in the update section 66). Other cache rows in the set can be updated to reflect the change.For example, if the filled / hit cache row becomes the MRU, the position of any other cache row can be moved from the current MRU to the previous position of the filled / hit cache row by one position towards the LRU. If the filled / hit cache row is moved to a position other than the MRU in the replacement data, any cache row that has a position different from the current position of the filled / hit cache row can be moved towards the LRU.

[0034] In insertion section 64, the previous state is zero because the cache row is being filled into cache 32. For this section, request types other than non-timely (NT) require updating the substitute data to perform the fill at MRU for critical cache rows. If the fill involves a prefetch request (data or instruction) and the criticality value is a non-critical state, the fill is performed at LRU position N, which is close to the LRU position but not the LRU position itself. For example, N can be about 25% of the distance between the LRU and the MRU above the LRU. For instance, if cache 32 has 8 paths, 25% above the LRU would be 2 positions above the LRU. If cache 32 has 16 paths, 25% above the LRU would be 4 positions above the LRU.If the fill is for a demand retrieval (instruction or data) and the criticality value is non-critical, the fill is performed at LRU position L (near the center of the substitute data area). For example, if the cache has 32 paths and 8 paths, L can be in the range of positions 4 to 6 in various embodiments, assuming the LRU position is numbered 0. If the cache has 16 paths, L can be in the range of 6 to 8. If the fill is for a non-critical demand retrieval, the LRU position of the fill can be at position M, close to the LRU but less than N.

[0035] In the embodiment of Fig. 3. The update of the replacement data on a hit of the cache rows can be independent of the criticality value of the cache row. Other embodiments can take criticality into account during the update. If the hit request is a demand call (instruction or data) and the cache row was a pre-fed cache row, the LRU position can remain unchanged (NC), but the pre-fed trace bit can be left unaided for the cache row, so that the next time the cache row is hit it will be a demand call. If the hit request is a demand call (instruction or data) and the hit cache row was an NT demand or a demand call (instruction or data), the hit cache row can be made the MRU. If the hit request is a data pre-fed, the hit cache row can be placed at N (near the LRU).If the hit request is a pre-instruction call, the hit cache line is made into the MRU. If the hit request is an NT request, the hit cache can be position N.

[0036] With reference to Fig. Figure 4 shows a flowchart illustrating the operation of an embodiment of the criticality control circuit 34 for selecting a sacrificial cache row to be emitted when a cache miss is detected. Although the blocks are shown in a specific order for ease of understanding, other sequences can be used. Blocks can be executed in parallel using combinational logic in the criticality control circuit 34. Blocks, combinations of blocks, and / or the flowchart as a whole can be executed over multiple clock cycles. The criticality control circuit 34 can be configured to perform the following actions: Fig. To implement the 4 shown operations.

[0037] If at least one invalid cache entry exists in the record indexed by the cache miss (decision block 70, "yes" branch), the criticality control circuit 34 can select the LRU-nearest invalid entry (block 72). An invalid entry can be a cache row location (e.g., path) that is not currently storing a cache row. The LRU-nearest invalid entry can be the invalid entry that is invalid and has a position that—compared to the positions of the other invalid entries—is closest to the LRU position in the substitute data. The LRU-nearest invalid entry can be located at the LRU position.

[0038] If no invalid entries are present in the set (decision block 70, "no" branch), the criticality control circuit 34 can select a valid entry as the sacrifice. In a typical LRU policy, the LRU entry might be selected. However, in this embodiment, the criticality control circuit 34 can retain the critical cache rows with a certain probability. Accordingly, a falsified pseudorandom selection can be generated (e.g., based on a linear feedback shift register or LFSR and the desired probability) (block 74). Based on the pseudorandom selection, the criticality control circuit 34 can selectively mask the critical cache rows against the selection (block 76). For example, if the falsified pseudorandom selection indicates an evaluation of the falsified test (e.g., "yes"), the critical cache rows might not be masked.If the falsified pseudorandom value indicates a different evaluation of the falsified test (e.g., "no"), the critical cache rows may be masked. This type of probability-based retention can also be referred to as a "falsified coin toss." The criticality control circuit 34 can select the LRU nearest valid, unmasked entry and can expel the cache block in that entry (block 78).

[0039] Fig. Figure 5 is a flowchart illustrating the operation of the criticality control circuit 34 for another embodiment of assigning criticality to a cache row that is populated in the LLC 14. Although the blocks are shown in a specific order for ease of understanding, other sequences can be used. Blocks can be executed in parallel using combinational logic in the criticality control circuit 34. Blocks, combinations of blocks, and / or the flowchart as a whole can be executed over multiple clock cycles. The criticality control circuit 34 can be configured to perform the operations described in Figure 5. Fig. To implement the 5 shown operations.

[0040] Similar to the embodiment of Fig. 2. The cache row can be critical if it is filled as a result of an MMU table traverse request (decision block 80, "yes" branch), a load at the head of LDQ 26 (decision block 82, "yes" branch), or an instruction fetch at the head of IC miss queue 22 (decision block 84, "yes" branch). In this embodiment, there are multiple levels of critical status. If the criticality provided by MCache 40 indicates a critical status (decision block 86, "yes" branch), criticality control circuit 34 can increase the critical status level based on the status provided by MCache 40 (block 88). If MCache 40 is reported as non-critical (decision block 86, "no" branch), either the cache line was previously non-critical or the cache line was a miss in MCache 40.In these cases, the criticality control circuit 34 can initialize the criticality value at the lowest level of the critical status (Block 90).

[0041] If the cache row in the current fill is not critical (decision blocks 80, 82, and 84, "no" branches), but the criticality value provided by MCache 40 is critical (decision block 92, "yes" branch), criticality control circuit 34 can retain the criticality value provided by MCache 40 (block 94). Otherwise (decision block 92, "no" branch), criticality control circuit 34 can initialize the criticality value with a non-critical status (block 96).

[0042] Fig. Figure 6 is a flowchart illustrating the operation of an embodiment of the criticality control circuit 34 for updating the substitute data for a set based on filling a cache row into the cache 32 (an insertion of a cache row). Although the blocks are shown in a specific order for ease of understanding, other sequences can be used. Blocks can be executed in parallel using combinational logic in the criticality control circuit 34. Blocks, combinations of blocks, and / or the flowchart as a whole can be executed over multiple clock cycles. The criticality control circuit 34 can be configured to perform the operations described in Figure 6. Fig. 6. To implement the operation shown.

[0043] If the cache row being populated has a high critical state (e.g., a critical state that is not the lowest of the critical states in an embodiment) (decision block 100, "yes" branch), the cache row can be inserted at the MRU position in the substitute data (block 102). If the cache row has a critical state (e.g., the lowest critical state) (decision block 100, "no" branch and decision block 104, "yes" branch), the criticality control circuit 34 can be configured to insert the cache row at the highest possible position in the substitute data (closest to the MRU), but below the positions of cache rows with a high critical state.Therefore, if one or more cache rows with a high critical status are present in the replacement data (decision block 106, "yes" branch), the criticality control circuit 34 can insert the cache row at the highest position that is lower than the cache rows with a high critical status (block 108). Otherwise, the cache row can be inserted at the MRU position (decision block 106, "no" branch, and block 102).

[0044] If the cache row being filled is not critical (decision blocks 100 and 104, "no" branches) and the fill is due to a prefetch (instruction or data) (decision block 110, "yes" branch), the prefetch can be inserted at N near the LRU (block 112), similar to above with respect to Fig. 3 discussed. In one embodiment, instruction precalls can be placed at a lower LRU position than data precalls, but both can be placed near the LRU position. Alternatively, instruction precalls can be placed at a higher LRU position than data precalls, but both can be placed near the LRU, or the same LRU position can be used for both precall types. If the non-critical cache line is not a precall but an NT request (decision block 114, "yes" branch), the cache line can be inserted at position M, which in this embodiment is greater than N but near the LRU (block 116). If the non-critical cache row is a requirement (decision block 114, "No" branch) and there are critical cache rows (decision block 106, "Yes" branch), the non-critical cache row can be inserted below the critical cache rows (block 108).If there are no critical cache lines in the set (decision block 106, "no" branch), the non-critical cache line can be inserted at the MRU position (block 102).

[0045] The circuit, represented by decision block 106 and blocks 102 and 108, can provide a dynamic insertion point for certain cache rows, thereby preventing a “priority reversal” in the surrogate data where critical cache rows could be moved down towards the LRU position by less critical cache rows in the surrogate data.

[0046] Fig. Figure 7 is a flowchart illustrating the operation of an embodiment of the criticality control circuit 34 for updating the substitute data for a set based on a hit on a cache line in the criticality control circuit 34 (a cache line promotion). Although the blocks are shown in a specific order for ease of understanding, other sequences can be used. Blocks can be executed in parallel using combinational logic in the criticality control circuit 34. Blocks, combinations of blocks, and / or the flowchart as a whole can be executed over multiple clock cycles. The criticality control circuit 34 can be configured to perform the operations described in Figure 7. Fig. 7. To implement the operation shown.

[0047] If the hit cache row has any level of critical status (decision block 120, "yes" branch), criticality control circuit 34 can update the cache row at the MRU position (block 122). If the hit cache row is not critical (decision block 120, "no" branch) and the hit cache row is an untouched prefetch request (decision block 124, "yes" branch), criticality control circuit 34 can leave the surrogate data position unchanged but can reset the prefetch bit (block 126). If the hit request is a demand or data pre-fetch (decision block 128, "yes" branch), the criticality control circuit 34 can maintain the priority of the critical cache rows by promoting the hit cache row to the highest substitute data position below the critical cache rows (decision block 130, "yes" branch and block 132).If there are no critical cache lines in the set, the hit cache line can be made into the MRU (decision block 130, "No" branch and block 122). If the hit request is an NT request (decision block 134, "Yes" branch), the hit cache line can be updated to position P, which is close to the LRU, provided the hit cache line is an untouched prefetch, meaning its position remains unchanged (block 136). If the hit request is not an NT request (and none of the other request types mentioned above), the request can be an instruction prefetch, and the hit cache line can be updated into the MRU (block 138).

[0048] Similar to the above regarding Fig. As discussed in section 6, the switching logic represented by decision block 130 can provide a dynamic substitute data update to prevent priority reversal between non-critical cache rows and critical cache rows. The embodiment of Fig. Option 6 can allow different levels of critical cache rows to be reordered in the replacement data, but can keep the non-critical cache rows below the critical cache rows in the replacement data.

[0049] With reference to Fig. Figure 8 shows a flowchart illustrating the operation of an embodiment of the criticality control circuit 34 for selecting a sacrificial cache row to be emitted when a cache miss is detected. Although the blocks are shown in a specific order for ease of understanding, other sequences can be used. Blocks can be executed in parallel using combinational logic in the criticality control circuit 34. Blocks, combinations of blocks, and / or the flowchart as a whole can be executed over multiple clock cycles. The criticality control circuit 34 can be configured to perform the following actions: Fig. To implement the 8 operations shown.

[0050] If at least one invalid entry is present in the set (decision block 140, "yes" branch), the criticality control circuit 34 can mask all valid entries and select the LRU nearest unmasked (invalid) entry (block 142). If all entries are valid (decision block 140, "no" branch), the criticality control circuit 34 can determine a falsified pseudo-random selection, similar to the above with respect to Fig. 4 (Block 144) is discussed. Based on the pseudorandom selection, the criticality control circuit 34 can selectively mask all critical cache rows (Block 146). If at least one unmasked, valid entry is found (Decision Block 148, "Yes" branch), the criticality control circuit 34 can select the LRU nearest unmasked entry (Block 142). If no entry is found (Decision Block 148, "No" branch), the criticality control circuit 34 can unmask the lowest level of critical cache rows while continuing to mask the higher critical cache rows (Block 150). If at least one unmasked, valid entry is found (Decision Block 152, "Yes" branch), the criticality control circuit 34 can select the LRU nearest unmasked entry (Block 142).If no entry is found (decision block 152, "no" branch), the criticality control circuit 34 can unmask all critical cache lines (block 154) and select the nearest unmasked entry to the LRU (block 142).

[0051] Fig. Figure 9 is a flowchart illustrating the operation of another embodiment of the criticality control circuit 34 for selecting a sacrificial cache row to be emitted when a cache miss is detected. Although the blocks are shown in a specific order for ease of understanding, other sequences can be used. Blocks can be executed in parallel using combinational logic in the criticality control circuit 34. Blocks, combinations of blocks, and / or the flowchart as a whole can be executed over multiple clock cycles. The criticality control circuit 34 can be configured to perform the operations described in Figure 9. Fig. To implement the operation shown in section 9.

[0052] The embodiment of Fig. 9 can use multiple rigged pseudorandom selection procedures based on different probabilities to selectively mask or unmask different subsets of the critical state until a victim is selected. Similar to the embodiment of Fig. If there is at least one invalid entry in the set (decision block 160, "yes" branch), criticality control circuit 34 can mask all valid entries and select the LRU nearest unmasked (invalid) entry (block 162). If all entries are valid (decision block 160, "no" branch), criticality control circuit 34 can determine an initial falsified pseudorandom selection based on a first probability, similar to the above with respect to Fig. 4 is discussed (Block 164). If the selection is "yes" (Decision Block 168, "Yes" branch), the criticality control circuit 34 can mask all critical cache rows (Block 168) and determine if at least one valid, unmasked entry was found (Decision Block 170). If so (Decision Block 170, "Yes" branch), the criticality control circuit 34 can select the LRU nearest unmasked entry (Block 162). If not (Decision Block 170, "No" branch), or if the selection was "no" (Decision Block 166, "No" branch), the criticality control circuit 34 can determine a second rigged pseudorandom selection based on a second probability (Block 172).If the selection is "yes" (decision block 174, "yes" branch), the criticality control circuit 34 can mask critical cache rows, except for the lowest critical state (block 176), and determine if at least one valid, unmasked entry has been found (decision block 178). If so (decision block 178, "yes" branch), the criticality control circuit 34 can select the nearest unmasked entry to the LRU (block 162). If not (decision block 178, "no" branch), or if the selection was "no" (decision block 174, "no" branch), the criticality control circuit 34 can continue with similar iterations, masking fewer of the highest critical state levels until an entry is found (block 180) or until all critical rows are unmasked. Once an entry has been found, the criticality control circuit can select the nearest unmasked entry to the LRU (block 162).

[0053] Implementations that employ dynamic replacement data updates to preferentially retain critical cache rows closer to the MRU than other cache rows can successfully maintain the cache rows in LLC 14. However, once the critical cache rows are no longer useful, these same properties can increase the difficulty of replacing them with recently accessed, non-critical cache rows.

[0054] As mentioned above, LLC 14 can be configured to preferentially retain cache rows identified as critical by their respective criticality values, rather than those not identified as critical, when selecting a sacrificial cache row for replacement. LLC 14 can also be configured to select replacement data for the sacrificial cache row based on data held separately from the criticality values ​​(while still taking the criticality values ​​into account). However, if criticality control circuit 34 detects one or more indications that at least some of the cache rows identified as critical are no longer critical, criticality control circuit 34 can be configured to stop preferentially retaining cache rows based on these one or more indications.Viewed from another perspective, the criticality control circuit 34 can accelerate the release of the cache lines identified as critical based on one or more specifications (compared to the retention that would be applied before the one or more specifications are detected). For example, in one embodiment, the criticality control circuit 34 can be configured to ignore the criticality values ​​when the sacrificial cache line is selected, either to terminate the preferably retention of the critical cache lines or to accelerate their release.

[0055] Fig. Figure 10 is a flowchart illustrating the operation of an embodiment of the criticality control circuit 34 for accelerating the emission of critical cache lines that are no longer in use. Although the blocks are shown in a specific order for ease of understanding, other sequences can be used. Blocks can be executed in parallel using combinational logic in the criticality control circuit 34. Blocks, combinations of blocks, and / or the flowchart as a whole can be executed over multiple clock cycles. The criticality control circuit 34 can be configured to perform the following actions in Figure 10: Fig. To implement the 10 operations shown.

[0056] When critical cache rows are no longer accessed (e.g., the accessing thread(s) has / have completed execution), the critical cache rows in the surrogate data can eventually migrate to the LRU position. Accordingly, the criticality control circuit 34 can monitor the hit rates for critical cache rows located in the N nearest LRU positions (block 190). N can be chosen in any desired way. For example, N can be approximately one-quarter of the number of paths in a set. Furthermore, if snoop copy-backs of cache rows from LLC 14 increase (i.e., snoops cause the cache rows to be redirected to another processor 10A-10N), the thread accessing the critical cache rows may have migrated to a different cluster of processors 10A-10N coupled to a different LLC 14 in the system (in Fig. (1 not shown). Accordingly, the criticality control circuit 34 can monitor the snoop rates that cause cache rows to be forwarded to other agents in the system (not back to the memory controller 16) (Block 192). In various embodiments, the criticality control circuit 34 can monitor snoop forwards from critical cache blocks only or from all cache blocks. Another factor that can be monitored is derived coprocessor requests (requests from coprocessor 12) (Block 194).

[0057] If the cache hit rates, as captured by the monitoring represented by block 190, are less than a threshold (decision block 196, "yes" branch), criticality control circuit 34 can ignore the criticality values ​​for victim selection and LRU insertion and promotion (block 198). This allows cache rows to be treated equally regardless of their critical / non-critical status. Similarly, if the snoop forwarding rates exceed a threshold (decision block 200, "yes" branch), criticality control circuit 34 can ignore the criticality values ​​for victim selection and LRU insertion and promotion (block 198). Similarly, if derived coprocessor requests increase (decision block 202, "yes" branch), criticality control circuit 34 can ignore the criticality values ​​for victim selection and LRU insertion and promotion (block 198).

[0058] Another factor that can be used is when the capacity in MCache 40 available to the 10A-10N / LLC 14 processors falls below a threshold (e.g., as indicated by the CPU capacity reading from monitoring circuit 42) (Decision Block 204, "Yes" branch, the criticality control circuit 34 treats all levels of critical state as the lowest critical state (Block 206)). If none of the above is true (Decision Blocks 196, 200, 202, and 204, "No" branches), the criticality control circuit 34 can retain the use of criticality values ​​in sacrifice selection and LRU insertion and promotion (Block 208).

[0059] Thus, in this embodiment, the one or more specifications can include a cache hit rate below a threshold for cache rows at a multitude of the longest unused positions in the surrogate data, with criticality values ​​indicating a critical status. The one or more specifications can include a rate at which snoop hits occur in the cache and, in response to the snoop hit exceeding a threshold, cause the forwarding of a corresponding cache row. In systems that include a coprocessor coupled to the cache and configured to execute coprocessor instructions issued to the coprocessor by the one or more processors, the one or more specifications can include memory requests issued to the cache by the coprocessor.The criticality control circuit 34 can be configured to derive coprocessor memory requirements based on prefetch requests generated by the one or more processors specified as coprocessor prefetch requests. As mentioned above, the MCache 40 can provide an indication of the capacity in the second cache that can be mapped to the data from the LLC 14, and the control circuit is configured to override the multitude of criticality levels with the lowest of the multitude of criticality levels based on the indication that the capacity is lower than a threshold.

[0060] In one embodiment, a method may comprise: assigning criticality values ​​to cache rows in a cache, wherein a given criticality value corresponds to a given cache row; during the selection of a sacrificial cache row for replacement, preferably retaining cache rows identified as critical based on the corresponding criticality values ​​over cache rows not identified as critical, the selection further being based on replacement data kept separate from the criticality values ​​by the cache; recognizing one or more indications that at least some of the cache rows identified as critical are no longer critical; and ignoring the criticality values ​​for the sacrificial selection and replacement data update based on the one or more indications.For example, in one embodiment, the method further comprises monitoring a cache hit rate for cache rows located at a multitude of the longest unused positions in the surrogate data and exhibiting criticality values ​​indicating a critical status, with one or more of these values ​​based on the cache hit rate being below a threshold. In another embodiment, the method further comprises monitoring a rate at which snoop hits occur in the cache and cause the forwarding of a corresponding cache row in response to the snoop hit, with one or more of these values ​​based on the snoop hit rate being above a threshold.In one embodiment, the one or more specifications comprise memory requests issued by a coprocessor to the cache, wherein the coprocessor is coupled to the cache and configured to execute coprocessor instructions issued to the coprocessor by one or more processors. The method may further comprise deriving the coprocessor memory requests based on prefetch requests generated by one or more processors, which are specified as coprocessor prefetch requests.In one embodiment, the method may further comprise providing a capacity indication from a second cache in the second cache, which is mappable to data from the cache, wherein the criticality values ​​are non-critical and indicate a plurality of criticality levels; and overwriting the plurality of criticality levels with the lowest of the plurality of criticality levels based on the fact that the capacity indication is lower than a threshold.

[0061] Fig. Figure 11 is a flowchart illustrating the operation of an embodiment of the criticality control circuit 34 and the MCache 40 for inserting emitted cache lines from the LLC 14 into the substitute data of the MCache 40. Although the blocks are shown in a specific order for ease of understanding, other sequences can be used. Blocks can be executed in parallel using combinational logic in the criticality control circuit 34 and / or the MCache 40. Blocks, combinations of blocks, and / or the flowchart as a whole can be executed over multiple clock cycles. The criticality control circuit 34 / MCache 40 can be configured to perform the operations described in Figure 11. Fig. To implement the 11 operations shown.

[0062] If the exposed cache row is a critical cache row (decision block 210, "yes" branch), the criticality control circuit 34 can generate the insertion instruction to the MCache 40 to insert the cache row at the MRU position (block 212). Alternatively, the MCache 40 can detect the critical status of the cache row and insert the cache row at the MRU position. If the cache row is not critical (decision block 210, "no" branch), the criticality control circuit 34 can generate and index the MCache insertion LUT 36 based on the CPU capacity telemetry data (block 214). For example, the CPU capacity telemetry data can specify the average number of paths of the MCache 40 available for cache rows from the 10A-10N / LLC 14 processors. The index can be generated based on the fact that the average number of paths lies in different areas.For example, up to one-eighth of the number of paths, one-eighth to one-quarter of the number of paths, one-quarter to one-half of the number of paths, and more than one-half of the number of paths can be the index for a two-bit insertion hint. The criticality control circuit 34 can generate the insertion hint from the indexed entry in the MCache insertion LUT 36 (block 216).

[0063] When the insertion control circuit and LUT 38 receive the emitted cache block, the insertion hint can be used as an index for LUT 38, and the insertion position can be read from the table (block 218). The insertion control circuit 38 can modify the insertion position based on whether MCache paths are disabled to save power. That is, each disabled path occupies an LRU position in the substitute data, as it cannot be used. If the insertion position would be in one of the N LRU positions, where N is the number of disabled paths, the insertion position can be incremented to N (block 220). MCache 40 can assign an entry for the cache line and update the entry with the cache line (block 222), whereby the cache line, if any, emitted from MCache 40 is written to memory 18 if it has been modified with respect to the copy in memory 18.The MCache 40 can update the replacement data to display the assigned entry at the insertion position (block 224).

[0064] In one embodiment, the MCache 40 can also support a dynamic insertion position for non-critical cache rows from the 10A-10N / LLC14 processors. For example, the MCache 40 can determine the MRU-nearest non-critical cache row (not including the cache row for which the insertion point is detected), which is referred to in this paragraph as position H. If no non-critical cache rows are present, the MCache 40 can insert the cache row at the adjusted insertion position described in the preceding paragraph. However, if valid non-critical cache rows exist in the MCache 40 and the inserted cache row is already located at a position closer to the MRU than position H, the MCache 40 can insert the cache row at the position closer to the MRU than position H. Otherwise, the MCache 40 can insert the cache row at position H.The MCache 40 can update the replacement data to display the assigned entry at the insertion position (block 224).

[0065] Fig. Figure 12 is a block diagram of an embodiment of a system that includes a system-on-a-chip (SOC) 300 coupled to a memory 18. As the name suggests, the components of the SOC 300 can be integrated on a single semiconductor substrate as an integrated circuit “chip”. In the illustrated embodiment, the components of the SOC 300 include a processor cluster 304, another processor cluster 206, one or more peripheral components such as the peripheral components 308A-308B (referred to as “peripherals”), the memory controller 16, and a communication structure 312. The components 304, 306, 308A-308B, and 16 can all be coupled to the communication structure 312. The memory controller 16 can be coupled to the memory 18 during use. In some embodiments, more than one memory controller may be present, coupled to a corresponding memory.In such embodiments, the memory address space can be allocated in any desired way via the memory controllers. In the illustrated embodiment, the processor cluster 304 can include a plurality of processors (P) 10A-10N. The processors 10A-10N can form the central processing units (CPUs) of the SOC 300. The processor cluster 304 can further include one or more coprocessors (e.g., the coprocessor 12 in ). Fig. 12) include. The 304 processor cluster can also include the LLC 14. The 306 processor cluster can be similar to the 304 processor cluster. Thus, the SOC 300 can be an implementation of the in Fig. 1 of the systems shown.

[0066] Memory controller 16 can generally include the switching logic for receiving memory operations from the other components of the SOC 300 and accessing memory 18 to complete those operations. Memory controller 12 can be configured to access any type of memory 18. For example, memory 18 can be static RAM (SRAM) or dynamic RAM (DRAM), such as synchronous DRAM (SDRAM), including double-data-rate DRAM (DDR, DDR2, DDR3, DDR4, etc.). Low-performance / mobile versions of DDR DRAM can be supported (e.g., LPDDR, mDDR, etc.). Memory controller 16 can include queues for memory operations to order (and potentially reorder) the operations and to deliver the operations to memory 18.The memory controller 16 may further include data buffers to store write data awaiting write operation to memory and read data awaiting return to the memory source. In some embodiments, the memory controller 16 may include a memory cache (MCache) 40 to store recently accessed memory data. In SOC implementations, for example, the MCache 40 may reduce power consumption in the SOC by avoiding re-accessing data from memory 16 when it is expected to be accessed again soon. In some cases, the MCache 40 may also be referred to as a system cache, in contrast to private caches such as the LLC 14 or caches in the 10A-10N processors, which serve only specific components. Additionally, in some embodiments, a system cache need not reside within the memory controller 16.

[0067] The 308A-308B peripherals can be any set of additional hardware functionality included in the SOC 300. For example, the 308A-308B peripherals can include video peripherals, such as one or more graphics processing units (GPUs), an image signal processor configured to process image capture data from a camera or other image sensor, video encoders / decoders, scalers, rotators, display controllers, mixers, etc. The peripherals can include audio peripherals, such as microphones, speakers, microphone and speaker interfaces, audio processors, digital signal processors, mixers, etc. The peripherals can include interface controllers for various interfaces outside the SOC 100, including interfaces such as Universal Serial Bus (USB), Peripheral Component Interconnect (PCI), including PCI Express (PCIe), serial and parallel ports, etc.The connection to the external device in . Fig. 12 is illustrated by a dashed arrow extending outside the SOC 300. Peripherals can include network peripherals, such as Media Access Controllers (MACs). Any set of hardware can be included.

[0068] The Communication Structure 312 can be any communication link and any protocol used for communication between the components of the SOC 300. The Communication Structure 312 can be bus-based, including shared bus configurations, crossbar configurations, and hierarchical buses with bridges. The Communication Structure 312 can also be packet-based and can be hierarchical with bridges, crossbars, point-to-point, or other connections.

[0069] It should be noted that the number of components of the SOC 300 (and the number of subcomponents of those that are in Fig. Figure 12 shows how, for example, the number of processors 10A-10N in each processor cluster 304 and 306 can vary from embodiment to embodiment. Additionally, the number of processors 10A-10N in one processor cluster 304 can differ from the number of processors 10A-10N in the other processor cluster 306. There can be more or fewer of each component / subcomponent than shown in Figure 12. Fig. 12 shown number.

[0070] Based on the foregoing, in one embodiment a system may comprise one or more processors configured to issue memory requests to access a memory system; and a cache coupled to the one or more processors and configured to temporarily store data from the memory system for access by the one or more processors. The cache may include a control circuit configured to assign criticality values ​​to the cache rows based on a variety of factors at a time when the cache rows are being filled into the cache. During the filling of a given cache row, the control circuit may be configured to represent a given cache row at a selected position in the substitute data for the cache based on the criticality value assigned to the given cache row.The control circuit can be configured to select a sacrificial cache row to be emitted from the cache based on the replacement data. The control circuit can also be configured to selectively prevent the selection of cache rows with criticality values ​​indicating a critical status as sacrificial cache rows based on a probability.

[0071] In one embodiment, the system further comprises a second cache coupled to the cache and configured to temporarily store data from the storage system for the cache and for one or more other cache access agents in the system. The second cache is configured to store the sacrificial cache row and to maintain an indication of the criticality value assigned to the sacrificial cache row by the control circuit. In one embodiment, the system further comprises a memory controller configured to control one or more memory devices that constitute at least one section of the system memory, and the memory controller includes the second cache.In one embodiment, the second cache can be configured to provide the criticality value along with the sacrificial cache row in a fill to the cache based on another storage request that occurs after the sacrificial cache row is released from the cache. In another embodiment, the second cache is configured to maintain second backup data; and the second cache can be configured to release cache rows from the second cache based on the second backup data. An initial position of the sacrificial cache row in the second backup data can be based on the criticality value. In another embodiment, the system includes a monitoring circuit coupled to the second cache and configured to provide a capacity indication in the second cache that is mappable to data from the cache.The cache can be configured to generate an insertion hint to be transmitted with the sacrificial cache row based on the capacity specification. For example, the cache can include a table coupled to the control circuit that maps ranges of the capacity specification to the values ​​for the insertion hint. In one embodiment, the second cache includes a second table. The second cache can be configured to select an entry in the second table based on the insertion hint. The second table can be configured to output an insertion point indication from the selected entry. In one embodiment, the criticality status can include critical and non-critical. In another embodiment, the criticality status can further specify one or more criticality levels assigned to a critical cache row.In one embodiment, the control circuit can be configured to update the substitute data based on a request that encounters a second given cache row in the cache. The substitute data can be updated such that the position of a second entry storing the second given cache row is moved closer to a last accessed position, based on the criticality value assigned to the second given cache row and the criticality values ​​of other cache rows represented in the substitute data.For example, if the criticality value assigned to the second given cache row is lower than the criticality values ​​of one or more other cache rows represented in the substitute data, the control circuit can be configured to update the substitute data so that the second given cache row is represented in a second position below positions occupied by one or more other cache rows.

[0072] In one embodiment, the control circuit is configured to monitor a cache hit rate for cache rows at a variety of low positions in the surrogate data and to display criticality values ​​with a critical status. The control circuit can be configured to ignore the criticality values ​​for victim selection and surrogate data updates based on the cache hit rate being below a threshold. In another embodiment, the control circuit can be configured to monitor a snoop hit rate for snoops that cause cache rows to be redirected from the cache. The control circuit can be configured to ignore the criticality values ​​for victim selection and surrogate data updates based on the snoop hit rate exceeding a threshold. computer system

[0073] Next, referring to Fig. Figure 13 shows a block diagram of an embodiment of a system 700. In the illustrated embodiment, the system 700 includes at least one instance of a system-on-a-chip (SOC) 706, which is coupled to one or more peripheral devices 704 and an external memory 702. A power supply unit (PMU) 708 is provided, which supplies supply voltages to the SOC 706 and one or more supply voltages to the memory 702 and / or the peripheral devices 704. In some embodiments, more than one instance of the SOC may be included (and more than one memory 702 may also be included). In one embodiment, the memory 702 may be the Fig. 1 and Fig. 12 illustrated memory 18 include. The SOC 706 can, in one embodiment, include an instance of the in Fig. 12 illustrated SOC 300.

[0074] The 704 peripherals can include any desired switching logic, depending on the type of system 700. For example, in one embodiment, the system 704 can be a mobile device (e.g., a personal digital assistant (PDA), a smartphone, etc.), and the 704 peripherals can include devices for various types of wireless communication, such as Wi-Fi, Bluetooth, cellular networks, global positioning systems, etc. The 704 peripherals can also include additional storage, including RAM, solid-state storage, or disk storage. The 704 peripherals can include user interface devices, such as a display screen, including touchscreens or multi-touchscreens, keyboards or other input devices, microphones, speakers, etc.In other embodiments, the System 700 can be any type of computing system (e.g., desktop personal computer, laptop, workstation, nettop, etc.).

[0075] The 702 external memory can include any type of memory. For example, the 702 external memory can be SRAM, dynamic RAM (DRAM), such as synchronous DRAM (SDRAM), double data rate SDRAM (DDR, DDR2, DDR3, etc.), RAMBUS DRAM, low-power versions of DDR DRAM (e.g., LPDDR, mDDR, etc.), and so on. The 702 external memory can include one or more memory modules to which the memory devices are attached, such as single inline memory modules (SIMMs), dual inline memory modules (DIMMs), and so on. Alternatively, the 702 external memory can include one or more memory devices attached to the 706 SoC in a chip-on-chip or package-on-package implementation.

[0076] As illustrated, System 700 is shown to have applications in a wide range of fields. For example, System 700 can be used as part of the chips, switching logic, components, etc., of a desktop computer 710, laptop computer 720, tablet computer 730, cellular or mobile phone 740, or television 750 (or a set-top box coupled with a television). Also illustrated are a smartwatch and a health monitoring device 760. In some embodiments, a smartwatch may include a variety of functions relating to general data processing. For example, a smartwatch may provide access to email, a cellular service, a user calendar, and so on. In various embodiments, a health monitoring device may be a dedicated medical device or otherwise include dedicated health-related functionality.For example, a health monitoring device can monitor a user's vital signs, track a user's proximity to other users for the purpose of maintaining epidemiological distance, track contacts, facilitate communication with an emergency service in the event of a health emergency, and so on. In various embodiments, the aforementioned smartwatch may include some or no health monitoring-related functions. Other wearable devices are also being considered, such as devices worn around the neck, devices that can be implanted in the human body, glasses designed to provide an augmented and / or virtual reality experience, and so forth.

[0077] The System 700 can also be used as part of a cloud-based service (770). For example, the devices mentioned above and / or other devices can access computing resources in the cloud (i.e., remotely located hardware and / or software resources). Furthermore, the System 700 can be used in one or more other household devices besides those mentioned above. For example, devices within the home can monitor and detect conditions that require attention. For example, various devices within the home (e.g., a refrigerator, a cooling system, etc.) can monitor the device's status and provide an alert to the homeowner (or, for example, a repair facility) should a specific event be detected.Alternatively, a thermostat can monitor the temperature in the apartment and automate settings of a heating / cooling system based on a history of responses to different conditions by the apartment owner. Also in . Fig. Figure 13 illustrates the application of System 700 in various modes of transport. For example, System 700 can be used in the control and / or entertainment systems of airplanes, trains, buses, rental cars, passenger vehicles, watercraft, from private boats to cruise ships, (rental or owned) scooters, and so on. In various cases, System 700 can be used to provide automated guidance (e.g., self-driving vehicles), general system control, and other things. Any number of these other embodiments are possible and are being considered. It should be noted that the in Fig. The 13 illustrated devices and applications are for illustrative purposes only and are not intended to be limiting. Other devices are possible and will be considered. Computer-readable storage medium

[0078] Now, referring to Fig. Figure 14 shows a block diagram of an embodiment of a computer-readable storage medium 800. In general terms, a computer-accessible storage medium can include any storage medium that can be accessed by a computer during use to provide instructions and / or data to the computer. For example, a computer-accessible storage medium can include storage media such as magnetic or optical media, e.g., disks (fixed or portable), tapes, CD-ROM, DVD-ROM, CD-R, CD-RW, DVD-R, DVD-RW, or Blu-ray. Storage media can further include volatile or non-volatile storage media, such as RAM (e.g., synchronous dynamic RAM (SDRAM), Rambus DRAM (RDRAM), static RAM (SRAM), etc.), ROM, or flash memory. The storage media can be physically enclosed within the computer to which the storage media provide instructions / data.Alternatively, the storage media can be connected to the computer. For example, the storage media can be connected to the computer via a network or a wireless connection, such as network storage. The storage media can be connected via a peripheral interface, such as the Universal Serial Bus (USB). Generally, the computer-accessible storage medium can store 800 data in a non-transitory manner, where non-transitory in this context can refer to the fact that the instructions / data are not transmitted on a signal. For example, non-transitory storage can be volatile (and lose the stored instructions / data in response to a shutdown), or it can be non-volatile.

[0079] The computer-accessible storage medium 800 in Fig. 14 can store a database 804 that is representative of the SOC 300. In general, the database 804 can be a database that can be read by a program and used directly or indirectly to construct the hardware that comprises the SOC 300. For example, the database can be a behavioral-level or register-transfer-level (RTL) description of the hardware functionality in a high-level design (HDL) language such as Verilog or VHDL. The description can be read by a synthesis tool that can synthesize it to generate a netlist containing a list of gates from a synthesis library. The netlist includes a set of gates that also represent the functionality of the hardware that comprises the SOC 300. The netlist can then be placed and routed to generate a data set that describes geometric shapes to be applied to masks.The masks can then be used in various semiconductor fabrication steps to produce a semiconductor circuit or circuits equivalent to the SOC 300. Alternatively, the database 804 on the computer-accessible storage medium 800 can be, as desired, the netlist (with or without the synthesis library) or the data set.

[0080] While the computer-accessible storage medium 800 stores a representation of the SOC 300, other embodiments can, as desired, store a representation of any section of the SOC 100, including any subset of the data contained within. Fig. The 12 components shown are included. Furthermore, the 804 database can incorporate the 10A-10N processors, the 12 coprocessor, or both, as shown in Fig. 1 shown, represents, and can further represent the LLC 14 and / or the memory controller 16. The database 804 can represent any section of the foregoing.

[0081] Various embodiments are considered, as illustrated in the following numbered examples: 1. System, comprehensive: one or more processors configured to issue memory requests for accessing a storage system; and a cache that is coupled to the one or more processors and configured to temporarily store data from the storage system for access by the one or more processors, wherein: The cache includes a control circuit configured to assign criticality values ​​to cache rows, where a given criticality value corresponds to a given cache row; the cache is configured to preferentially retain cache rows identified as critical based on the relevant criticality values ​​over cache rows not identified as critical when selecting a victim cache row for replacement, furthermore the cache is configured to select the victim cache row according to replacement data kept separate from the criticality values ​​by the cache; the control circuit is configured to detect one or more indications that at least some of the cache lines identified as critical are no longer critical; and The control circuit is configured to stop preferentially retaining the cache lines based on one or more specifications. 2. System according to Example 1, wherein the control circuit is configured to monitor a cache hit rate for cache rows located at a multitude of the longest unused positions in the substitute data and exhibiting criticality values ​​indicating a critical status, and wherein one of the one or more indications is based on the cache hit rate being below a threshold. 3. System according to Example 1 or 2, wherein the control circuit is configured to monitor a rate at which snoop hits occur in the cache and cause a forwarding of a corresponding cache row in response to the snoop hit, and wherein one or more of the specifications is based on the snoop hit rate being above a threshold. 4. System according to any of Examples 1 to 3, further comprising a coprocessor coupled to the cache and configured to execute coprocessor instructions issued to the coprocessor by the one or more processors, and wherein the one or more instructions comprise memory requests issued to the cache by the coprocessor. 5. System according to Example 4, wherein the control circuit is configured to derive the coprocessor memory requirements based on prefetch requests generated by the one or more processors, which are specified as coprocessor prefetch requests. 6. System according to any of the preceding examples, further comprising a second cache coupled to the cache, wherein the second cache is configured to provide an indication of the capacity in the second cache that is attributable to the data from the cache, and wherein the criticality values ​​indicate non-critical and a plurality of criticality levels, and wherein the control circuit is configured to override the plurality of criticality levels with the lowest of the plurality of criticality levels based on the indication that the capacity is lower than a threshold. 7. System according to one of the preceding examples, wherein the control circuit is configured to terminate the preferred retention at least partially by ignoring the criticality values ​​for victim selection and replacement data update. 8. System, comprehensive: one or more processors configured to issue memory requests for accessing a storage system; and a cache that is coupled to and configured with one or more processors to temporarily store data from a storage system for access by one or more processors, wherein the cache includes a control circuit and wherein: the control circuit is configured to assign criticality values ​​to the cache rows, where a given criticality value corresponds to a given cache row; the cache is configured to preferentially retain cache rows identified as critical based on the relevant criticality values ​​over cache rows not identified as critical when selecting a victim cache row as a replacement, furthermore the cache is configured to select the victim cache row according to replacement data kept separate from the criticality values ​​by the cache; the control circuit is configured to detect one or more indications that at least some of the cache lines identified as critical are no longer critical; and The control circuit is configured to accelerate the emission of cache lines identified as critical based on one or more specifications. 9. System according to Example 8, wherein the control circuit is configured to monitor a cache hit rate for cache rows located at a variety of the longest unused positions in the surrogate data and exhibiting criticality values ​​indicating a critical status, and wherein one of the one or more indications is based on the cache hit rate being below a threshold. 10. System according to Example 8 or 9, wherein the control circuit is configured to monitor a rate at which snoop hits occur in the cache and to cause a forwarding of a corresponding cache row in response to the snoop hit, and wherein one or more of the specifications are based on the snoop hit rate being above a threshold. 11. System according to one of Examples 8 to 10, wherein the one or more specifications comprise memory requests issued by a coprocessor to the cache, wherein the coprocessor is coupled to the cache and is configured to execute coprocessor instructions issued to the coprocessor by the one or more processors. 12. System according to Example 11, wherein the control circuit is configured to derive the coprocessor memory requests based on prefetch requests generated by the one or more processors, which are specified as coprocessor prefetch requests. 13. System according to any of Examples 8 to 12, further comprising a second cache configured to provide an indication of the capacity in the second cache that is mappable to data from the cache, wherein the criticality values ​​indicate non-critical and a plurality of criticality levels, and wherein the control circuit is configured to override the plurality of criticality levels with the lowest of the plurality of criticality levels based on the fact that the capacity indication is lower than a threshold. 14. System according to one of Examples 8 to 13, wherein the control circuit is configured to accelerate the emission at least partially by ignoring the criticality values ​​for victim selection and replacement data update. 15. Procedure, comprehensive: Assigning criticality values ​​to cache rows in a cache, where a given criticality value corresponds to a given cache row; during the selection of a victim cache row for replacement, preferably retaining cache rows that have been identified as critical based on the relevant criticality values, over cache rows that have not been identified as critical, the selection further being based on replacement data that is kept separate from the cache and the criticality values; Detecting one or more indications that at least some of the cache lines identified as critical are no longer critical; and Ignoring criticality values ​​for victim selection and replacement data update based on one or more pieces of information. 16. The method according to Example 15, further comprising monitoring a cache hit rate for cache rows located at a multitude of the longest unused positions in the surrogate data and exhibiting criticality values ​​indicating a critical status, and wherein one or more of the indications is based on the cache hit rate being below a threshold. 17. A method according to Example 15 or 16, further comprising monitoring a rate at which snoop hits occur in the cache and cause a forwarding of a corresponding cache row in response to the snoop hit, and wherein one or more of the specifications are based on the snoop hit rate being above a threshold. 18. Method according to any of Examples 15 to 17, wherein the one or more specifications comprise memory requests issued by a coprocessor to the cache, wherein the coprocessor is coupled to the cache and is configured to execute coprocessor instructions issued to the coprocessor by one or more processors. 19. The method according to Example 18, further comprising deriving the coprocessor memory requirements based on prefetch requests generated by one or more processors, which are specified as coprocessor prefetch requests. 20. Procedure according to one of Examples 15 to 19, furthermore comprising: Providing a reference from a second cache of the capacity in the second cache that is mappable to data from the cache, where the criticality values ​​indicate non-critical and a variety of criticality levels; and Overriding the multitude of criticality levels with the lowest of the multitude of criticality levels based on the fact that the capacity specification is lower than a threshold.

Claims

[1] Institution, encompassing: one or more processors (10A - 10N) configured to issue memory requests for accessing a memory system (18); and a cache switching logic (32) configured to cache data from the storage system (18) for access by the one or more processors (10A - 10N), wherein: the cache switching logic (32) includes a control switching logic (34) that is configured to: Assigning criticality values ​​to cache rows, where a given criticality value indicates that the corresponding cache row is not critical or has one of several criticality levels; and Adjusting the criticality value of a cache row in response to a criticality event; where the cache switching logic (32) to select a sacrificial cache line for replacement is configured to: Masking cache lines that have one or more higher criticality levels; and Selecting the victim cache row from unmasked cache rows based on access frequency data managed separately from criticality values ​​by the cache switching logic (32). [2] Device according to claim 1, further comprising: a loading queue switching logic (26); where the criticality event corresponds to a charge accessing the cache row that is the oldest pending charge in the load queue switching logic (26). [3] Device according to claim 1, further comprising: Memory management switching logic (30), wherein the criticality event corresponds to an access to the cache line requested by the memory management switching logic (30). [4] Device according to claim 1, further comprising: an error queue switching logic (22) for retrieval requests that fail in an instruction cache (20); where the criticality event corresponds to a retrieval request accessing the cache row that is the oldest pending retrieval request in the fault queue switching logic (22). [5] Device according to claim 1, wherein the adaptation is one of the following adaptations: Assigning a lowest criticality value to replace a non-critical value; and an increase in the criticality of an already critical criticality value. [6] Device according to claim 1, wherein masking cache rows with one or more higher criticality levels includes masking lower criticality levels until a candidate-victim cache row is found. [7] Device according to claim 1, wherein: the device includes a control switching logic (34) configured to monitor the capacity of a higher-level cache that can be allocated data from the cache switching logic (32); and The cache switching logic is configured to assign a cache row to the lowest criticality level based on the monitored capacity being below a threshold. [8] Device according to claim 1, wherein the cache switching logic (32) to assign a cache entry for a non-critical cache row is configured to determine an artificial access frequency data value for the non-critical cache row based on access frequency data values ​​of cached critical cache rows. [9] Device according to claim 8, wherein the artificial access frequency value is less recent than an access frequency value of a recently used cached critical cache line. [10] Device according to claim 1, wherein: the device includes a control switching logic (34) configured to monitor the capacity of a higher-level cache that can be allocated data from the cache switching logic (32); and the cache switching logic (32) to assign a cache entry for a non-critical cache row is configured to determine an artificial access frequency data value for the non-critical cache row based on the monitored capacity. [11] Device according to claim 1, wherein the device is a computing device further comprising: a display control switching logic; and a network interface switching logic. [12] Device according to claim 1, wherein the device is an integrated circuit. [13] Procedures, including: Output, by a computing device, of memory requests for access to a storage system (18); Temporarily storing, by the computing device, data from the storage system (18); Assigning, by the computing device, criticality values ​​to cache rows, wherein a given criticality value indicates that the corresponding cache row is not critical or has one of several criticality levels; Adjusting, by the computing device, a criticality value of a cache row in response to a criticality event; Select, by the computing device, a sacrificial cache line to replace, including: Masking cache lines that have one or more higher criticality levels; and Selecting the victim cache row from unmasked cache rows based on access frequency data that is managed separately from criticality values. [14] Method according to claim 13, wherein the criticality event corresponds to a charge accessing the cache row that is the oldest pending charge in a charge queue (26). [15] Non-transitory computer-readable medium on which instructions of a hardware description programming language are stored which, when processed by a computing system, program the computing system to generate a computer model, the model representing a hardware circuit which includes: one or more processors (10A - 10N) configured to issue memory requests for accessing a memory system (18); and a cache switching logic (32) configured to cache data from the storage system (18) for access by the one or more processors (10A - 10N), wherein: the cache switching logic (32) includes a control switching logic (34) that is configured to: Assigning criticality values ​​to cache rows, where a given criticality value indicates that the corresponding cache row is not critical or has one of several criticality levels; and Adjusting the criticality value of a cache row in response to a criticality event; where the cache switching logic (32) to select a sacrificial cache line for replacement is configured to: Masking cache lines that have one or more higher criticality levels; and Selecting the victim cache row from unmasked cache rows based on access frequency data managed separately from criticality values ​​by the cache switching logic (32).

Images