CRITICALITY-INFORMED CACHING POLICIES

Abstract

System (700), comprising: one or more processors (10A-10N) configured to issue memory requests to access a memory system (702); and a cache (14) coupled to the one or more processors (10A-10N) and configured to cache data from the memory system (702) for access by the one or more processors (10A-10N), wherein: the cache (14) comprises a control circuit (34) configured to assign criticality values ​​to cache lines based on a plurality of factors at a time the cache lines are filled into the cache (14); the control circuit (34) is configured to, during the fill of a given cache line, represent the given cache line at a selected position in replacement data for the cache (14) based on the criticality value assigned to the given cache line; the control circuit (34) is configured to select a victim cache line to be evicted from the cache (14) based on the replacement data; and the control circuit (34) is configured to determine, according to a probability-based mode selection for a given cache eviction operation, whether to proceed according to a first victim selection mode or a second victim selection mode, wherein the first victim selection mode prevents the selection of cache lines with a critical status as victims and the second victim selection mode allows the selection of a cache line with a critical status as victims.

Description

BACKGROUNDTechnical field

[0001] The embodiments described herein relate to caches in computer systems, and more particularly to caching policies. Description of the state of the art

[0002] Caches have long been used in digital systems to reduce effective memory latency by capturing a copy of data accessed by a processor, coprocessor, or other digital device in a cache memory located locally on the device. The cache memory may be smaller than the main memory system and can be optimized for low latency (whereas the main memory system is often optimized for storage density, at the expense of latency). Accordingly, the cache memory itself can reduce latency. Additionally, the cache memory can be located locally on the device, and thus latency can be reduced because there is no transport delay to the memory controller / main memory system and back to the device. Furthermore, the cache can be private to the device or a small number of devices (e.g.,Processor / coprocessor cluster), and thus the competition for bandwidth with the cache can be reduced compared to main memory.

[0003] While caches reduce effective memory latency, they represent limited storage and are therefore vulnerable to misses (which cause a fill from memory to the cache to obtain the data, in addition to delivering the data to the requesting device if the miss involves a read request, or performing the update if the miss involves a write request). The fill allocates memory in the cache (e.g., a cache line or a cache block). The allocation may cause other data in the cache to be replaced (also known as evicting a cache line from the cache). There are a variety of replacement policies for selecting which cache line to evict based on the cache geometry.For example, set-associative caches have memory arranged as a two-dimensional array of cache lines: a "row" is selected based on a subset of the cache line's memory address (called a set), and the row encloses a plurality of cache lines that are the "columns" of the array (called ways). When a cache miss is detected and a fill is initiated, one of the ways is assigned to the fill. A popular replacement policy for set-associative caches is the least recently used (LRU) policy. With LRU, accesses to the cache lines in a set are tracked from most recently used (MRU) to least recently used (LRU).Typically, when a cache line is accessed, it is updated to the MRU, and the cache lines between the cache line's former rank and the previous MRU are adjusted. The LRU cache line can be selected for replacement when a cache miss occurs.

[0004] Here, US 2011 / 0145506 A1 discloses, in one embodiment, a cache memory with cache lines that store the importance of data as a weight, taking into account the coherence status and the timeliness of use. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The following detailed description refers to the accompanying drawings, which are now briefly described. Fig. 1 is a block diagram of one embodiment of a portion of a system. Fig. 2 is a flowchart illustrating a criticality determination for one embodiment. Fig. 3 is a table illustrating LRU insertion and update in one embodiment of a Fig. 1 illustrates the last-level cache (LLC). Fig. Figure 4 is a flowchart illustrating one embodiment of victim selection in the LLC. Fig. 5 is a flowchart illustrating criticality determination for another embodiment. Fig. 6 is a flowchart illustrating an LRU insertion in the LLC for one embodiment. Fig. 7 is a flowchart illustrating LRU promotion in the LLC for one embodiment. Fig. 8 is a flowchart illustrating victim selection for one embodiment. Fig. 9 is a flowchart illustrating victim selection for another embodiment. Fig. Figure 10 is a flowchart illustrating eviction acceleration for cache lines marked as critical. Fig. 11 is a flowchart illustrating LRU insertion for cache lines in the memory cache for one embodiment. Fig. 12 is a block diagram of one embodiment of a system on a chip (SOC). Fig. 13 is a block diagram of various embodiments of a computer system. Fig. 14 is a block diagram of one embodiment of a computer-accessible storage medium.

[0006] While embodiments described in this disclosure are susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and are described in detail herein. It is to be understood, however, that the drawings and the detailed description thereof are not intended to limit the embodiments of the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents, and alternatives that are within the spirit and scope of the appended claims. The headings used herein are for organizational purposes only and should not be used to limit the scope of the description. DETAILED DESCRIPTION OF EMBODIMENTS

[0007] While the LRU replacement policy often provides good performance (e.g., cache hit rates remain high, effectively reducing memory latency), there are cases where performance may be limited. For example, if contention for cache lines is high and evictions occur frequently, some cache lines may be evicted, which, when accessed again, cause a greater performance loss to the requesting device than other cache lines. For example, if a number of operations on the requesting device depend on the data in the cache line directly or indirectly through other operations, the requesting device may stall while waiting for the data. Other cache lines with fewer dependencies may be less critical to performance. The LRU policy has no way of reflecting differences in the criticality of cache lines.

[0008] According to the present invention, a system according to claim 1, a method according to claim 15 and a cache according to claim 20 are provided.

[0009] In one embodiment, the system, including one or more processors and a cache coupled to the one or more processors, may categorize cache lines according to one or more criticality levels based on one or more criteria measured at the time the cache lines are filled into the cache. The criteria may be selected to attempt to identify the cache lines that, if they represent a cache miss, have a greater impact on the performance of the processors than other cache lines. Each cache line may have a criticality value indicating its criticality level. For example, the critical value may indicate a non-critical status or a critical status. In one embodiment, the critical status may also have multiple criticality levels, as described in more detail below.In another embodiment, the critical status may be a single level indicating critical, as opposed to the non-critical status.

[0010] The cache may implement a replacement policy that uses the criticality values ​​of the cache lines as a factor. For example, an LRU policy may be used, but the policy may be modified to account for the criticality of different cache lines. Cache lines with a criticality value indicating a critical status ("critical cache lines") may be inserted into the LRU replacement data at the MRU position, while cache lines with criticality values ​​indicating a non-critical status ("non-critical cache lines") may be inserted at lower positions in the data (e.g., closer to the LRU position). In one embodiment, criticality values ​​may also influence the updating of the LRU replacement data. While LRU is used as an example replacement policy, other embodiments may implement different replacement policies.For example, a variety of pseudo-LRU policies can be used that approximate LRU by making simplifications to make the policy easier to implement, particularly in large set-associative caches. Additionally, random replacement policies can be used, and criticality can be used to reduce the probability that critical lines are selected. Least Frequently Used policies can be used, and critical lines can be selectively retained in a similar manner as described below for LRU. Last in, first out or first in, first out policies can be used, and critical cache lines can be at least partially exempted from LIFO or FIFO replacement. Any of these policies can be modified to account for criticality.

[0011] In one embodiment, the system may include one or more additional cache levels between the aforementioned cache and the system memory. For example, a memory cache implemented at the memory controller that controls the system memory may be used. Cache line criticality values ​​may be exchanged between caches when the cache lines are evicted and retrieved, maintaining the criticality values ​​while the cache lines remain cached in the cache hierarchy. Once the cache line is evicted from the cache hierarchy (and thus the data exists only in system memory), the criticality value may be lost.

[0012] Fig. 1 is a block diagram of one embodiment of a system including a plurality of processors 10A-10N, a coprocessor 12, a last-level cache (LLC) 14, a memory controller 16, and a memory 18. Processors 10A-10N and coprocessor 12 are coupled to LLC 14, which is coupled to memory controller 16, which is further coupled to memory 18. Processor 10N is illustrated in more detail, and other processors like processor 10A may be similar. Processor 10N may include an instruction cache (ICache) 20, an instruction cache miss queue (IC-miss queue) 22, an execution core 24 including a load queue (LDQ) 26, a data cache (DCache) 28, and a memory management unit (MMU) 30. The LLC 14 may include a cache 32, a criticality control circuit 34, and a memory cache insertion lookup table (MCache insertion LUT) 36.The memory cache 16 may include an insertion control circuit and LUT 38, an MCache 40, and a monitor circuit 42.

[0013] The ICache 20 may store instructions fetched by the processor 10N for execution by the execution core 24. If a fetch in the ICache 20 fails, the instruction cache line fetch may be placed in the IC miss queue 22 and transferred to the LLC 14 as a fill request for the ICache 20. Instructions executed by the execution core 24 may include load instructions (in short, loads). The loads may attempt to read data from the DCache 28 and, if a load in the DCache 28 fails, may be transferred to the LLC 14 as a fill request for the DCache 28. The loads transferred to the LLC 14 may remain in the LDQ 26 to wait for data.

[0014] The MMU 30 may provide address translations for instruction fetch addresses and load / store addresses, including translation buffers (TLBs) that may be local to the ICache 20 and the execution core 24. The MMU 30 may optionally include one or more level 2 TLBs (L2 TLBs) as well as table walk circuitry to perform translation table reads to obtain a translation for an address that misses the TLBs. The MMU 30 may transfer the table walk reads to the LLC 14. In one embodiment, the MMU 30 may access the DCache 28 for a potential cache hit on the table walk reads before transferring them to the LLC 14 and may not transfer the reads to the LLC 14 if they hit the DCache 28.In other embodiments, the page table data is not cached in the DCache 28 and the MMU 30 may transfer table traversal reads to the LLC 14.

[0015] The LLC 14 includes the cache 32, which may have any capacity and configuration. Memory requests from the processors 10A-10N and the coprocessor 12 may be checked for a hit in the cache 32, and in the case of a hit, the data may be returned as a fill to the ICache 20, the DCache 28, or the MMU 30. If the memory request results in a miss in the cache 32, the LLC 14 may transmit a memory request to the memory controller 16 and, in response to the memory controller 16 returning a fill to the LLC 14, may return the fill to the requesting processor 10A-10N or the coprocessor 12. The LLC 14 may also fill the data into the cache 32 in the case of a miss.In general, "data" is used herein in a general sense, such that it refers to both instructions fetched by processors 10A-10N for execution and data read / written by the processors as a result of the execution of the instructions (e.g., operand data and result data), particularly when referring to cache lines of data.

[0016] Additionally, LLC 14 may assign a criticality value to the cache line at the time of fill to processor 10A-10N / coprocessor 12. Criticality control circuitry 34 may determine the criticality value and update cache 32 with the criticality value. For example, the cache tags in cache 32 may include a field for the criticality value. The critical value may indicate a non-critical status or a critical status. As mentioned above, in some embodiments, there may be more than one level of critical status. Criticality control circuitry 34 may also determine the level of the critical condition.

[0017] Criticality control circuitry 34 may consider a variety of factors when assigning criticality values ​​to cache lines. For example, criticality control circuitry 34 is coupled to MMU 30, IC miss queue 22, and LDQ 26. In particular, fills that are for table walk requests may be categorized as critical. A TLB miss is likely to affect additional instruction fetches or load / store requests because a translation involves a fairly large amount of data, and code sequences tend to access data located near other recently fetched data. For example, a page may be 4 kilobytes, 16 kilobytes, or even larger, such as 1 megabyte or 2 megabytes. Any page size may be used. Additionally, a load that is at the head of LDQ 26 when the fill for the load occurs may be the oldest outstanding load in processor 10N.Thus, it is likely that the load is blocking the departure of other completed instructions or that a number of instructions are blocked due to dependence on the load data (either directly or indirectly). Fills for loads located at the head of the LDQ 25 may be assigned a critical status. Similarly, if there is a fill for an instruction fetch request and it is the oldest fetch request in the IC miss queue 22 (e.g., located at the head of the IC miss queue 22), it is likely that the fetching of the instructions is blocked by waiting for instructions. Such instruction fetches may be assigned a critical status. Other embodiments may include additional factors within a given processor 10A-10N or subsets of the foregoing factors and other factors, as desired.In one embodiment, requests from coprocessor 12 may also be assigned a critical status. For example, one embodiment of coprocessor 12 may not include a cache, and thus LLC 14 is the first level of caching available to coprocessor 12. Cache lines that have not been assigned a critical status may be assigned a non-critical status.

[0018] In one embodiment, the criticality values ​​assigned to cache lines may be retained while the cache lines remain valid in the cache hierarchy. The criticality value is assigned by criticality control circuitry 34 and then propagated with the cache line when it is evicted from cache 32 and transferred to memory controller 16, where it may be cached in MCache 40. If the evicted cache line is placed in MCache 40 after being evicted from cache 32, the criticality value may be retained. If the evicted cache line is not placed in MCache 40 after being evicted from cache 32, memory controller 16 may drop the criticality value and write the data to memory 18. There may be a variety of factors that affect whether or not an evicted cache line is cached in MCache 40.The MCache 40 is shared with other components of the system, and the MCache 40 may have quotas on how much data from a given component can be cached. If the LLC 14 has exceeded the quota, the evicted cache line may not be cached. Alternatively, the evicted cache line may be cached, and another LLC cache line cached in the MCache 40 may be evicted.

[0019] Subsequently, when a cache line previously cached by LLC 14 is accessed again by LLC 14, MCache 40 may provide the cache line to cache 32 as a fill, and the criticality value previously associated with the cache line may also be provided. Criticality control circuit 34 may assign the previous criticality value provided to the cache line by MCache 40 unless other factors from processor 10A-10N that generated a re-access to the cache line indicate an update to a critical status or to a higher level of critical status. For example, a non-critical cache line from MCache 40 may be filled into LLC 14 with a non-critical status unless it is assigned critical status at the time of the fill for re-access (e.g.,If the fill is for a load at the head of the LDQ 26, an instruction fetch at the head of the IC miss queue 22, or an MMU table walk request, a critical cache line from the MCache 40 may be filled as critical. In embodiments implementing multiple criticality status levels, a critical cache line from the MCache 40 that is also currently indicated as critical via the above factors (head of the LDQ 26, head of the IC miss queue 22, or MMU request) may be assigned a higher level of critical status by the criticality control circuit 34.

[0020] In one embodiment, cache lines evicted from LLC 14 may be cached in MCache 40 and may be inserted at a selected position in the replacement data of the affected set of MCache 40. If the evicted cache line is a critical cache line, it may be inserted at the MRU position. If the evicted cache line is a non-critical cache line, it may be inserted at a position lower than the MRU (closer to the LRU). In one embodiment, the insertion point for non-critical cache lines may be dynamic. For example, the insertion point may be based on the amount of cache capacity in MCache 40 occupied by cache lines of LLC 14. The memory controller 16 may include a monitoring circuit 42 that monitors the capacity of the MCache 40 associated with the CPU and provides the information (“CPU capacity”) to the criticality control circuit 34.Criticality control circuitry 34 may use the CPU capacity value as an index to the MCache insertion LUT 36 and may read an insertion hint from the indexed entry when an evicted cache line is transferred to memory controller 16. The insertion hint may be used as an index to a LUT 38 in memory controller 16, and the associated insertion control logic may adjust the insertion point (e.g., if a portion of the cache is disabled, the insertion point should be within the currently used LRU positions). MCache 40 may insert the evicted cache block at the insertion point.

[0021] Accordingly, in this embodiment, cooperative lookup tables can be used to determine the insertion point for evicted cache lines in the MCache 40 for non-critical cache lines. The LUTs can be programmable, allowing software to optimize performance as desired.

[0022] The CPU capacity value may be measured in any desired manner. In one embodiment, the CPU capacity may indicate the average number of MCache ways occupied by cache lines from LLC 14. In another embodiment, an approximate percentage of cache capacity may be provided.

[0023] As previously mentioned, the cache 32 may include a field (e.g., in the cache tag) for the criticality value. The MCache 40 may similarly include a field in the cache tag for the criticality value. In another embodiment, the MCache 40 may include a data set identifier (DSID) for each cache line that identifies related cache lines according to one or more criteria. Generally, cache blocks with the same DSID may be from the same source component (e.g., the LLC 14 or another component of the system, such as a Fig. 1 peripheral component not shown). The DSID may be stored in a field in the tag. The DSID may be used to distinguish non-critical and critical cache lines (e.g., by using one DSID for non-critical cache lines and a different DSID for critical cache lines, or multiple DSIDs for different levels of critical status in embodiments that use multiple levels of critical status). The MCache 40 may decode the DSID to determine the criticality value to be transmitted to the LLC 14 when providing a fill.

[0024] In one embodiment, processors 10A-10N may serve as the system's central processing unit (CPU). The system's CPU includes the one or more processors that execute the system's main control software, such as an operating system. Generally, software executed by the CPU during use may control the other components of the system to implement the system's desired functionality. Processors 10A-10N may also execute other software, such as application programs. The application programs may provide user functionality and may rely on the operating system for low-level device control, scheduling, memory management, etc. Accordingly, processors 10A-10N may also be referred to as application processors.

[0025] In general, a processor may include any circuitry and / or microcode configured to execute instructions defined in an instruction set architecture implemented by the processor. Processors may include processor cores implemented with other components as a system on a chip (SOC) on an integrated circuit or at other levels of integration. Processors may further include discrete microprocessors, as well as processor cores and / or microprocessors integrated into multi-chip module implementations; processors implemented as multiple integrated circuits; etc.

[0026] In one embodiment, coprocessor 12 may be configured to accelerate certain operations. For example, an embodiment is contemplated where a coprocessor performs large-scale matrix and vector manipulations (multiple operations per instruction). Coprocessor 12 may receive instructions transmitted from processors 10A-10N. That is, the instructions executed by coprocessor 12 ("coprocessor instructions") and the instructions executed by processors 10A-10N ("processor instructions") may be part of the same instruction set architecture and mixed in a code sequence fetched by the processor. Processor 10A-10N may decode the instructions and identify the coprocessor instructions for transmission to coprocessor 12 and execute processor instructions.Coprocessor 12 may receive coprocessor instructions from processor 10A-10N, decode the coprocessor instructions, and execute the coprocessor instructions. The coprocessor instructions may include load / store instructions to read memory data for operands and write result data to memory (in one embodiment, both may be performed in LLC 14).

[0027] It is noted that the number and type of different components of the system in Fig. 1 may vary from embodiment to embodiment. For example, any number of processors 10A-10N may be present. There may be more than one coprocessor 12, and if multiple coprocessors are included, there may be multiple instances of the same coprocessor and / or different types of coprocessors. There may be more than one memory controller 16, and if multiple memory controllers are included, memory space may be distributed among the memory controllers.

[0028] Note that various instructions, memory requests, etc., are referred to above as being younger or older than other instructions, requests, etc. A given operation may be younger than another operation if the given operation is derived from an instruction that is later in time than the instruction from which the other operation is derived in program order. Similarly, a given operation is older than another operation if the given operation is derived from an instruction that is earlier in time than the instruction from which the other operation is derived in program order.

[0029] The Fig. Figures 2 to 4 illustrate an embodiment in which criticality values ​​are either a critical or a non-critical status. Fig. 5 to 9 illustrate an embodiment in which the critical status has more than one criticality level. Fig. 10 illustrates a mechanism for accelerating the removal of critical cache lines from LLC 14 for one embodiment that may apply to both types of criticality values. Fig. 11 is a flowchart illustrating victim selection from LLC 14 based on the acceleration mechanism of Fig. 10 illustrates.

[0030] With reference to Fig. 2, a flowchart illustrating one embodiment of the criticality control circuit 34 for assigning a criticality value to a cache line being filled into the LLC 14 is shown. Although the blocks are shown in a particular order for ease of understanding, other orders may be used. Blocks may be executed in parallel in combinational logic within the criticality control circuit 34. Blocks, combinations of blocks, and / or the flowchart as a whole may be routed over multiple clock cycles. The criticality control circuit 34 may be configured to Fig. 2 to implement the operation shown.

[0031] If the fill is a cache line for an MMU table walk request (decision block 50, "Yes" branch), criticality control circuit 34 may assign a critical status for the criticality value associated with the cache line (block 52). If the fill is a cache line for a load operation located at the head of LDQ 26 (decision block 54, "Yes" branch), criticality control circuit 34 may assign a critical status for the criticality value associated with the cache line (block 52). If the fill is a cache line for an instruction cache miss that is at the head of the IC miss queue 22 (decision block 56, "yes" branch), the criticality control circuit 34 may assign a critical status for the criticality value associated with the cache line (block 52).If the fill is a cache line that has a critical status in the MCache (decision block 58, "Yes" branch), the criticality control circuit 34 may assign a critical status for the criticality value associated with the cache line (block 52). If none of the above criteria apply (decision blocks 50, 54, 56, 58, and 60, "No" branches), the criticality control circuit 34 may assign a non-critical status for the criticality value associated with the cache line. In one embodiment, the coprocessor requests from the coprocessor 12 may also be assigned a critical status. In another embodiment, the coprocessor requests may be assigned a non-critical status.

[0032] Fig. 3 is a table 62 illustrating the operation of one embodiment of criticality control circuitry 34 for updating the replacement data for a set based on a cache line fill into cache 32 (insertion section 64) and based on a cache hit for a processor request from a processor 10A-10N (update section 66). The update of the replacement data may be based on the request type, the previous state of the cache block, and the criticality value. The LRU column of the table indicates the position in the LRU ranking (from MRU to LRU) of the cache line being filled (in the insertion section 64) or the cache line being hit by a request (in the update section 66). Other cache lines in the set may be updated to reflect the change.For example, if the filled / hit cache line becomes MRU, the position of any other cache line from the current MRU to the previous position of the filled / hit cache line can be moved one position toward the LRU. If the filled / hit cache line is moved to a position other than MRU in the replacement data, any cache line with a position different from the current position of the filled / hit cache line can be moved toward the LRU.

[0033] In the insertion section 64, the previous state is zero because the cache line is being filled into cache 32. For this section, request types other than non-temporal (NT) require updating the replacement data to fill to the MRU for critical cache lines. If the fill involves a prefetch request (data or instruction) and the criticality value is a non-critical state, the fill is performed at LRU position N, which is close to the LRU position but not the LRU position itself. For example, N may be above the LRU by about 25% of the distance between the LRU and the MRU. For example, if cache 32 has 8 ways, 25% above the LRU would be 2 positions above the LRU. If cache 32 has 16 ways, 25% above the LRU would be 4 positions above the LRU.If the fill is for an on-demand fetch (instruction or data) and the criticality value is a non-critical state, the fill is performed at LRU position L (near the middle of the spare data area). For example, if the cache 32 has 8 ways, L may, in various embodiments, range from positions 4 to 6, assuming the LRU position is numbered 0. If the cache has 16 ways, L may range from 6 to 8. If the fill is for an NT on-demand fetch, the LRU position of the fill may be at position M, close to the LRU but less than N.

[0034] In the embodiment of Fig. 3, updating the replacement data on a cache line hit may be independent of the cache line's criticality value. Other embodiments may consider criticality in the update. If the hit request is an on-demand fetch (instruction or data) and the cache line was a prefetched cache line, the LRU position may be unchanged (NC), but the prefetch tracking bit may not be set for the cache line, so that the next time the cache line is hit, it will be an on-demand fetch. If the hit request is an on-demand fetch (instruction or data) and the hit cache line was an NT on-demand or on-demand fetch (instruction or data), the hit cache line may be made MRU. If the hit request is a data prefetch, the hit cache line may be placed at N (near the LRU).If the hit request is an instruction prefetch, the hit cache line is made MRU. If the hit request is an NT request, the hit cache line can be position N.

[0035] With reference to Fig. 4, a flowchart is shown illustrating the operation of one embodiment of the criticality control circuit 34 for selecting a victim cache line to be evicted when a cache miss is detected. Although the blocks are shown in a particular order for ease of understanding, other orders may be used. Blocks may be executed in parallel in combinational logic within the criticality control circuit 34. Blocks, combinations of blocks, and / or the flowchart as a whole may be routed over multiple clock cycles. The criticality control circuit 34 may be configured to Fig. 4 to implement the operation shown.

[0036] If at least one invalid cache entry exists in the set indexed by the cache miss (decision block 70, "Yes" branch), criticality control circuitry 34 may select the LRU-next invalid entry (block 72). An invalid entry may be a cache line location (e.g., way) that does not currently store a cache line. The LRU-next invalid entry may be the invalid entry that is invalid and has a position closest to the LRU position in the replacement data, compared to the positions of the other invalid entries. The LRU-next invalid entry may be located at the LRU position.

[0037] If there are no invalid entries in the set (decision block 70, "No" branch), criticality control circuitry 34 may select a valid entry as a victim. In a typical LRU policy, the LRU entry may be selected. However, in this embodiment, criticality control circuitry 34 may retain the critical cache lines with some probability. Accordingly, a corrupted pseudorandom selection may be generated (e.g., based on a linear feedback shift register or LFSR and the desired probability) (block 74). Based on the pseudorandom selection, criticality control circuitry 34 may selectively mask the critical cache lines from the selection (block 76). For example, if the corrupted pseudorandom selection indicates an evaluation of the corrupted test (e.g., "yes"), the critical cache lines may not be masked.If the corrupted pseudorandom value indicates a different evaluation of the corrupted test (e.g., "no"), the critical cache lines may be masked. This type of probability-based retention may also be referred to as a "corrupted coin toss." Criticality control circuitry 34 may select the next valid, unmasked entry in the LRU and may evict the cache block in that entry (block 78).

[0038] Fig. 5 is a flowchart illustrating the operation of the criticality control circuit 34 for another embodiment of assigning a criticality for a cache line being filled in the LLC 14. Although the blocks are shown in a particular order for ease of understanding, other orders may be used. Blocks may be executed in parallel in combinational logic in the criticality control circuit 34. Blocks, combinations of blocks, and / or the flowchart as a whole may be routed over multiple clock cycles. The criticality control circuit 34 may be configured to Fig. 5 to implement the operation shown.

[0039] Similar to the embodiment of Fig. 2, the cache line may be critical if the cache line is filled as a result of an MMU table walk request (decision block 80, "yes" branch), a load at the head of the LDQ 26 (decision block 82, "yes" branch), or an instruction fetch at the head of the IC miss queue 22 (decision block 84, "yes" branch). In this embodiment, there are multiple levels of critical status. If the criticality provided by the MCache 40 indicates critical status (decision block 86, "yes" branch), the criticality control circuit 34 may increment the critical status level from the status provided by the MCache 40 (block 88). If MCache 40 is indicated as non-critical (decision block 86, "No" branch), either the cache line was previously non-critical or the cache line was a miss in MCache 40.In these cases, the criticality control circuit 34 may initialize the criticality value at the lowest level of the critical status (block 90).

[0040] If the cache line in the current fill is not critical (decision blocks 80, 82, and 84, "No" branches), but the criticality value provided by the MCache 40 is a critical status (decision block 92, "Yes" branch), the criticality control circuit 34 may maintain the criticality value provided by the MCache 40 (block 94). Otherwise (decision block 92, "No" branch), the criticality control circuit 34 may initialize the criticality value to a non-critical status (block 96).

[0041] Fig. 6 is a flowchart illustrating the operation of one embodiment of the criticality control circuit 34 for updating the replacement data for a set based on a cache line fill into the cache 32 (a cache line insertion). Although the blocks are shown in a particular order for ease of understanding, other orders may be used. Blocks may be executed in parallel in combinational logic within the criticality control circuit 34. Blocks, combinations of blocks, and / or the flowchart as a whole may be routed over multiple clock cycles. The criticality control circuit 34 may be configured to Fig. 6 to implement the operation shown.

[0042] If the cache line being filled has a high critical status (e.g., a critical status that does not correspond to the lowest of the critical states, in one embodiment) (decision block 100, "Yes" branch), the cache line may be inserted at the MRU position in the replacement data (block 102). If the cache line has a critical status (e.g., the lowest critical status) (decision block 100, "No" branch and decision block 104, "Yes" branch), the criticality control circuit 34 may be configured to insert the cache line in the highest possible position in the replacement data (closest to the MRU), but below the positions of high critical status cache lines.Thus, if one or more high-criticality cache lines are present in the replacement data (decision block 106, "Yes" branch), criticality control circuit 34 may insert the cache line at the highest position, which is lower than the high-criticality cache lines (block 108). Otherwise, the cache line may be inserted at the MRU position (decision block 106, "No" branch, and block 102).

[0043] If the cache line being filled is not critical (decision blocks 100 and 104, “No” branches) and the fill is due to a prefetch (instruction or data) (decision block 110, “Yes” branch), the prefetch may be inserted at N near the LRU (block 112), similar to the above with respect to Fig. 3. In one embodiment, instruction prefetches may be placed at a lower LRU position than data prefetches, but both may be placed near the LRU position. Alternatively, instruction prefetches may be placed at a higher LRU position than data prefetches, but both may be placed near the LRU, or the same LRU position may be used for both prefetch types. If the non-critical cache line is not a prefetch but an NT request (decision block 114, "Yes" branch), the cache line may be inserted at position M, which in this embodiment is greater than N but near the LRU (block 116). If the non-critical cache line is a demand request (decision block 114, "No" branch) and there are critical cache lines (decision block 106, "Yes" branch), the non-critical cache line may be inserted below the critical cache lines (block 108).If there are no critical cache lines in the set (decision block 106, "No" branch), the non-critical cache line may be inserted at the MRU position (block 102).

[0044] The circuitry represented by decision block 106 and blocks 102 and 108 may provide a dynamic insertion point for certain cache lines, thereby preventing "priority inversion" in the replacement data when critical cache lines could be moved down toward the LRU position by less critical cache lines in the replacement data.

[0045] Fig. 7 is a flowchart illustrating the operation of one embodiment of the criticality control circuit 34 for updating the replacement data for a set based on a hit to a cache line in the criticality control circuit 34 (a cache line promotion). Although the blocks are shown in a particular order for ease of understanding, other orders may be used. Blocks may be executed in parallel in combinational logic in the criticality control circuit 34. Blocks, combinations of blocks, and / or the flowchart as a whole may be routed over multiple clock cycles. The criticality control circuit 34 may be configured to Fig. 7 to implement the operation shown.

[0046] If the hit cache line has any level of critical status (decision block 120, "Yes" branch), criticality control circuit 34 may update the cache line at the MRU location (block 122). If the hit cache line is not critical (decision block 120, "No" branch) and the hit cache line is an untouched prefetch request (decision block 124, "Yes" branch), criticality control circuit 34 may leave the replacement data location unchanged but may reset the prefetch bit (block 126). If the hit request is an on-demand or data prefetch (decision block 128, "Yes" branch), the criticality control circuit 34 may maintain the priority of the critical cache lines by promoting the hit cache line to the highest spare data position below the critical cache lines (decision block 130, "Yes" branch and block 132).If there are no critical cache lines in the set, the hit cache line may be made MRU (decision block 130, "No" branch, and block 122). If the hit request is an NT request (decision block 134, "Yes" branch), the hit cache line may be updated to position P, which is close to the LRU, provided the hit cache line is an untouched prefetch, meaning its position is unchanged (block 136). If the hit request is not an NT request (nor any of the other types of requests mentioned above), the request may be an instruction prefetch, and the hit cache line may be updated MRU (block 138).

[0047] Similar to the above with regard to Fig. 6, the logic represented by decision block 130 may provide dynamic replacement data updating to prevent priority inversion between non-critical cache lines and critical cache lines. The embodiment of Fig. 6 may allow different levels of critical cache lines to be reordered in the replacement data, but may keep the non-critical cache lines below the critical cache lines in the replacement data.

[0048] With reference to Fig. Figure 8 shows a flowchart illustrating the operation of one embodiment of the criticality control circuit 34 for selecting a victim cache line to be evicted when a cache miss is detected. Although the blocks are shown in a particular order for ease of understanding, other orders may be used. Blocks may be executed in combinational logic in the criticality control circuit 34 in parallel. Blocks, combinations of blocks, and / or the flowchart as a whole may be routed over multiple clock cycles. The criticality control circuit 34 may be configured to Fig. 8 to implement the operation shown.

[0049] If at least one invalid entry is present in the set (decision block 140, "Yes" branch), the criticality control circuit 34 may mask all valid entries and select the LRU-next unmasked (invalid) entry (block 142). If all entries are valid (decision block 140, "No" branch), the criticality control circuit 34 may determine a corrupted pseudorandom selection, similar to that described above with respect to Fig. 4 (block 144). Based on the pseudorandom selection, the criticality control circuit 34 may selectively mask all critical cache lines (block 146). If at least one unmasked, valid entry is found (decision block 148, "yes" branch), the criticality control circuit 34 may select the LRU-next unmasked entry (block 142). If no entry is found (decision block 148, "no" branch), the criticality control circuit 34 may unmask the lowest level of critical cache lines while continuing to mask the higher critical cache lines (block 150). If at least one unmasked, valid entry is found (decision block 152, "yes" branch), the criticality control circuit 34 may select the LRU-next unmasked entry (block 142).If no entry is found (decision block 152, "No" branch), the criticality control circuit 34 may unmask all critical cache lines (block 154) and select the LRU-next unmasked entry (block 142).

[0050] Fig. 9 is a flowchart illustrating the operation of another embodiment of the criticality control circuit 34 for selecting a victim cache line to be evicted when a cache miss is detected. Although the blocks are shown in a particular order for ease of understanding, other orders may be used. Blocks may be executed in parallel in combinational logic within the criticality control circuit 34. Blocks, combinations of blocks, and / or the flowchart as a whole may be routed over multiple clock cycles. The criticality control circuit 34 may be configured to Fig. 9 to implement the operation shown.

[0051] The embodiment of Fig. 9 may use multiple corrupted pseudorandom selection methods based on different probabilities to selectively mask or unmask different subsets of the critical state until a victim is selected. Similar to the embodiment of Fig. 8, if there is at least one invalid entry in the set (decision block 160, "Yes" branch), the criticality control circuit 34 may mask all valid entries and select the LRU-next unmasked (invalid) entry (block 162). If all entries are valid (decision block 160, "No" branch), the criticality control circuit 34 may determine a first corrupted pseudorandom selection based on a first probability, similar to that discussed above with respect to Fig. 4 (block 164). If the selection is "yes" (decision block 168, "yes" branch), the criticality control circuit 34 may mask all critical cache lines (block 168) and determine if at least one valid unmasked entry was found (decision block 170). If so (decision block 170, "yes" branch), the criticality control circuit 34 may select the LRU-next unmasked entry (block 162). If not (decision block 170, "no" branch), or if the selection was "no" (decision block 166, "no" branch), the criticality control circuit 34 may determine a second corrupted pseudorandom selection based on a second probability (block 172).If the selection is "yes" (decision block 174, "yes" branch), the criticality control circuit 34 may mask critical cache lines except for the lowest critical state (block 176) and determine if at least one valid unmasked entry was found (decision block 178). If so (decision block 178, "yes" branch), the criticality control circuit 34 may select the LRU-next unmasked entry (block 162). If not (decision block 178, "no" branch), or if the selection was "no" (decision block 174, "no" branch), the criticality control circuit 34 may continue with similar iterations, masking fewer of the highest levels of critical state, until an entry is found (block 180) or until all critical lines are unmasked. Once an entry is found, the criticality control circuit may select the LRU-next unmasked entry (block 162).

[0052] Embodiments that implement dynamic replacement data updates to preferentially maintain critical cache lines that are closer to the MRU than other cache lines may successfully maintain the cache lines in the LLC 14. However, once the critical cache lines are no longer useful, the same characteristics may increase the difficulty of replacing the critical cache lines with recently accessed cache lines that are not critical.

[0053] As mentioned above, during the selection of a victim cache line for replacement, the LLC 14 may be configured to preferentially retain cache lines identified as critical by the corresponding criticality values ​​over cache lines not identified as critical. The LLC 14 may be configured to select the victim cache line according to replacement data maintained by the cache separate from the criticality values ​​(and also taking the criticality values ​​into account). However, if the criticality control circuit 34 detects one or more indications that at least some of the cache lines identified as critical are no longer critical, the criticality control circuit 34 may be configured to discontinue preferentially retaining the cache lines based on the one or more indications.Viewed another way, criticality control circuitry 34 may accelerate the eviction of cache lines identified as critical based on the one or more indications (compared to the retention that would be applied prior to detecting the one or more indications). For example, in one embodiment, criticality control circuitry 34 may be configured to ignore the criticality values ​​when selecting the victim cache line to stop preferentially retaining or accelerate the eviction of the critical cache lines.

[0054] Fig. 10 is a flowchart illustrating the operation of one embodiment of criticality control circuit 34 for accelerating the eviction of critical cache lines that are no longer in use. Although the blocks are shown in a particular order for ease of understanding, other orders may be used. Blocks may be executed in parallel in combinational logic within criticality control circuit 34. Blocks, combinations of blocks, and / or the flowchart as a whole may be routed over multiple clock cycles. Criticality control circuit 34 may be configured to Fig. To implement the operation shown in Figure 10.

[0055] When critical cache lines are no longer accessed (e.g., the accessing thread(s) has / have completed execution), the critical cache lines in the replacement data may eventually migrate to the LRU position. Accordingly, the criticality control circuit 34 may monitor the hit rates for critical cache lines located in the N-nearest LRU positions (block 190). N may be selected in any desired manner. For example, N may be approximately one-quarter the number of ways in a set. Furthermore, if snoop copy-backs of cache lines from the LLC 14 increase (i.e., snoops cause the cache lines to be forwarded to another processor 10A-10N), the thread that accessed the critical cache lines may have migrated to another cluster of processors 10A-10N coupled to another LLC 14 in the system (in Fig. 1 not shown). Accordingly, criticality control circuitry 34 may monitor snoop rates that cause cache lines to be forwarded to other agents in the system (not back to memory controller 16) (block 192). Criticality control circuitry 34 may, in various embodiments, monitor snoop forwards only from critical cache blocks or from all cache blocks. Another factor that may be monitored is derived coprocessor requests (requests from coprocessor 12) (block 194).

[0056] If the cache hit rates detected via the monitoring represented by block 190 are less than a threshold (decision block 196, "Yes" branch), the criticality control circuit 34 may ignore the criticality values ​​during victim selection and LRU insertion and promotion (block 198). Thus, the cache lines may be treated equally regardless of critical / non-critical status. Similarly, if the snoop forwarding rates exceed a threshold (decision block 200, "Yes" branch), the criticality control circuit 34 may ignore the criticality values ​​during victim selection and LRU insertion and promotion (block 198). If derived coprocessor requests increase (decision block 202, "yes" branch), the criticality control circuit 34 may ignore the criticality values ​​during victim selection and LRU insertion and promotion (block 198).

[0057] Another factor that may be used is if the capacity in MCache 40 available to processors 10A-10N / LLC 14 falls below a threshold (e.g., as indicated by the CPU capacity indication from monitor circuit 42) (decision block 204, "Yes" branch), criticality control circuit 34 treats all levels of criticality as the lowest critical status (block 206). If none of the foregoing is true (decision blocks 196, 200, 202, and 204, "No" branches), criticality control circuit 34 may maintain the use of criticality values ​​in victim selection and LRU insertion and promotion (block 208).

[0058] Thus, in this embodiment, the one or more indications may include a cache hit rate below a threshold for cache lines at a plurality of least recently used locations in the replacement data and with criticality values ​​indicating a critical status. The one or more indications may include a rate at which snoop hits occur in the cache and, in response to the snoop hit being above a threshold, cause a corresponding cache line to be forwarded. In systems including a coprocessor coupled to the cache and configured to execute coprocessor instructions issued by the one or more processors to the coprocessor, the one or more indications may include memory requests issued by the coprocessor to the cache.Criticality control circuitry 34 may be configured to derive coprocessor memory requests based on prefetch requests generated by the one or more processors indicated as coprocessor prefetch requests. As mentioned above, MCache 40 may provide an indication of the capacity in the second cache attributable to data from LLC 14, and the control circuitry is configured to override the plurality of criticality levels with a lowest of the plurality of criticality levels based on the indication that the capacity is lower than a threshold.

[0059] In one embodiment, the method may comprise: assigning criticality values ​​to cache lines in a cache, wherein a given criticality value corresponds to a given cache line; during selection of a victim cache line for replacement, preferentially retaining cache lines identified as critical based on the corresponding criticality values ​​over cache lines not identified as critical, wherein the selection is further based on replacement data maintained by the cache separate from the criticality values; detecting one or more indications that at least some of the cache lines identified as critical are no longer critical; and ignoring the criticality values ​​for the victim selection and the replacement data update based on the one or more indications.For example, in one embodiment, the method further comprises monitoring a cache hit rate for cache lines located at a plurality of least recently used positions in the replacement data and having criticality values ​​indicating a critical status, and one of the one or more indications is based on the cache hit rate being below a threshold. In one embodiment, the method further comprises monitoring a rate at which snoop hits occur in the cache and cause a forwarding of a corresponding cache line in response to the snoop hit, and one of the one or more indications is based on the snoop hit rate being above a threshold.In one embodiment, the one or more indications comprise memory requests issued by a coprocessor to the cache, the coprocessor coupled to the cache and configured to execute coprocessor instructions issued by one or more processors to the coprocessor. The method may further comprise deriving the coprocessor memory requests based on prefetch requests generated by one or more processors indicated as coprocessor prefetch requests.The method may further comprise, in one embodiment, providing an indication from a second cache of the capacity in the second cache attributable to data from the cache, wherein the criticality values ​​indicate non-critical and a plurality of criticality levels; and overriding the plurality of criticality levels with a lowest of the plurality of criticality levels based on the indication of the capacity being lower than a threshold.

[0060] Fig. 11 is a flowchart illustrating the operation of one embodiment of the criticality control circuit 34 and the MCache 40 for inserting evicted cache lines from the LLC 14 into the replacement data of the MCache 40. Although the blocks are shown in a particular order for ease of understanding, other orders may be used. Blocks may be executed in combinational logic in the criticality control circuit 34 and / or the MCache 40 in parallel. Blocks, combinations of blocks, and / or the flowchart as a whole may be routed over multiple clock cycles. The criticality control circuit 34 / MCache 40 may be configured to Fig. To implement the operation shown in Figure 11.

[0061] If the evicted cache line is a critical cache line (decision block 210, "Yes" branch), the criticality control circuit 34 may generate the insertion hint to the MCache 40 to insert the cache line into the MRU location (block 212). Alternatively, the MCache 40 may detect the critical status of the cache line and insert the cache line into the MRU location. If the cache line is not critical (decision block 210, "No" branch), the criticality control circuit 34 may generate and index the MCache insertion LUT 36 based on the CPU capacity telemetry data (block 214). For example, CPU capacity telemetry data may indicate the average number of ways of MCache 40 available for cache lines from processors 10A-10N / LLC 14. The index may be generated based on the average number of ways being in different ranges.For example, up to one-eighth of the number of ways, one-eighth to one-quarter of the number of ways, one-quarter to one-half of the number of ways, and more than one-half of the number of ways may be the index for a two-bit insertion hint. Criticality control circuitry 34 may generate the insertion hint from the indexed entry in MCache insertion LUT 36 (block 216).

[0062] When the insertion control circuit and LUT 38 receive the evicted cache block, the insertion hint may be used as an index for the LUT 38, and the insertion position may be read from the table (block 218). The insertion control circuit 38 may modify the insertion position based on whether or not MCache ways are disabled to save power. That is, each disabled way occupies an LRU position in the spare data because it cannot be used. If the insertion position would be in one of the N LRU positions, where N is the number of disabled ways, the insertion position may be incremented to N (block 220). The MCache 40 may allocate an entry for the cache line and update the entry with the cache line (block 222), wherein the cache line evicted from the MCache 40, if any, is written to the memory 18 if it has been modified with respect to the copy in the memory 18.The MCache 40 may update the replacement data to indicate the assigned entry at the insertion position (block 224).

[0063] In one embodiment, MCache 40 may also support a dynamic insertion position for non-critical cache lines from processors 10A-10N / LLC14. For example, MCache 40 may determine the MRU-nearest non-critical cache line (not including the cache line for which the insertion point is detected), referred to in this paragraph as position H. If no non-critical cache lines are present, MCache 40 may insert the cache line at the adjusted insertion position described in the previous paragraph. However, if valid non-critical cache lines are present in MCache 40 and the inserted cache line is already at a position closer to the MRU than position H, MCache 40 may insert the cache line at the position closer to the MRU than position H. Otherwise, MCache 40 may insert the cache line at position H.The MCache 40 may update the replacement data to indicate the assigned entry at the insertion position (block 224).

[0064] Fig. 12 is a block diagram of one embodiment of a system including a system on a chip (SOC) 300 coupled to a memory 18. As the name suggests, the components of the SOC 300 may be integrated on a single semiconductor substrate as an integrated circuit "chip." In the illustrated embodiment, the components of the SOC 300 include a processor cluster 304, another processor cluster 206, one or more peripheral components such as peripheral components 308A-308B (collectively, "peripherals"), the memory controller 16, and a communications structure 312. Components 304, 306, 308A-308B, and 16 may all be coupled to the communications structure 312. The memory controller 16 may be coupled to the memory 18 during use. In some embodiments, there may be more than one memory controller coupled to a corresponding memory.In such embodiments, the memory address space may be allocated in any desired manner via the memory controllers. In the illustrated embodiment, the processor cluster 304 may include a plurality of processors (P) 10A-10N. The processors 10A-10N may form the central processing units (CPU(s)) of the SOC 300. The processor cluster 304 may further include one or more coprocessors (e.g., the coprocessor 12 in FIG. Fig. 12). The processor cluster 304 may further include the LLC 14. The processor cluster 306 may be similar to the processor cluster 304. Thus, the SOC 300 may include an implementation of the Fig. 1 shown system.

[0065] Memory controller 16 may generally include circuitry for receiving memory operations from the other components of SOC 300 and accessing memory 18 to complete the memory operations. Memory controller 12 may be configured to access any type of memory 18. For example, memory 18 may be static RAM (SRAM) or dynamic RAM (DRAM), such as synchronous DRAM (SDRAM), including double data rate DRAM (DDR, DDR2, DDR3, DDR4, etc.). Low-power / mobile versions of DDR DRAM may be supported (e.g., LPDDR, mDDR, etc.). Memory controller 16 may include store queues for ordering (and potentially reordering) the operations and providing the operations to memory 18.The memory controller 16 may further include data buffers to store write data waiting to be written to the memory and read data waiting to be returned to the source of the memory operation. In some embodiments, the memory controller 16 may include a memory cache (MCache) 40 to store recently accessed memory data. For example, in SOC implementations, the MCache 40 may reduce power consumption in the SOC by avoiding re-accessing data from the memory 16 when it is expected to be accessed again soon. In some cases, the MCache 40 may also be referred to as a system cache, in contrast to private caches, such as the LLC 14 or caches in the processors 10A-10N that serve only certain components. Additionally, in some embodiments, a system cache need not be located within the memory controller 16.

[0066] Peripherals 308A-308B may be any set of additional hardware functionality included in SOC 300. For example, peripherals 308A-308B may include video peripherals, such as one or more graphics processors (GPUs), an image signal processor configured to process image capture data from a camera or other image sensor, video encoders / decoders, scalers, rotators, display controllers, mixers, etc. The peripherals may include audio peripherals, such as microphones, speakers, microphone and speaker interfaces, audio processors, digital signal processors, mixers, etc. The peripherals may include interface controllers for various interfaces external to SOC 100, including interfaces such as Universal Serial Bus (USB), Peripheral Component Interconnect (PCI), including PCI Express (PCIe), serial and parallel ports, etc.The connection to the external device in . Fig. 12 is illustrated by a dashed arrow extending outward from the SOC 300. The peripherals may include network peripherals, such as media access controllers (MACs). Any set of hardware may be included.

[0067] The communication structure 312 can be any communication connection and protocol for communicating between the components of the SOC 300. The communication structure 312 can be bus-based, including shared bus configurations, crossbar configurations, and hierarchical buses with bridges. The communication structure 312 can also be packet-based and can be hierarchical with bridges, crossbars, point-to-point, or other connections.

[0068] It should be noted that the number of components of the SOC 300 (and the number of subcomponents of those included in Fig. 12, for example, how the number of processors 10A-10N in each processor cluster 304 and 306 may vary from embodiment to embodiment. In addition, the number of processors 10A-10N in one processor cluster 304 may differ from the number of processors 10A-10N in the other processor cluster 306. There may be more or fewer of each component / subcomponent than the number shown in Fig. 12 number shown.

[0069] Based on the foregoing, in one embodiment, a system may include one or more processors configured to issue memory requests to access a memory system; and a cache coupled to the one or more processors and configured to cache data from the memory system for access by the one or more processors. The cache may include control circuitry configured to assign criticality values ​​to cache lines based on a variety of factors at a time the cache lines are filled into the cache. During the fill of a given cache line, the control circuitry may be configured to represent a given cache line at a selected location in the replacement data for the cache based on the criticality value assigned to the given cache line.The control circuit may be configured to select a victim cache line to be evicted from the cache based on the replacement data. The control circuit may be configured to selectively prevent cache lines with criticality values ​​indicating a critical status from being selected as a victim cache line based on a probability.

[0070] In one embodiment, the system further comprises a second cache coupled to the cache and configured to cache data from the memory system for the cache and for one or more other cache access agents in the system. The second cache is configured to store the victim cache line and maintain an indication of the criticality value assigned to the victim cache line by the control circuitry. In one embodiment, the system further comprises a memory controller configured to control one or more memory devices forming at least a portion of the system memory, and the memory controller includes the second cache.In one embodiment, the second cache may be configured to provide the criticality value with the victim cache line in a fill to the cache based on another memory request that occurs after the victim cache line is evicted from the cache. In one embodiment, the second cache is configured to maintain second replacement data; and the second cache may be configured to evict cache lines from the second cache based on the second replacement data. An initial position of the victim cache line in the second replacement data may be based on the criticality value. In one embodiment, the system includes a monitor circuit coupled to the second cache and configured to provide a capacity indication in the second cache that is attributable to data from the cache.The cache may be configured to generate an insertion hint to be transmitted with the victim cache line based on the capacity indication. For example, the cache may include a table coupled to the control circuitry that maps ranges of the capacity indication to values ​​for the insertion hint. In one embodiment, the second cache includes a second table. The second cache may be configured to select an entry in the second table based on the insertion hint. The second table may be configured to output an insertion point indication from the selected entry. In one embodiment, the criticality status may include critical and non-critical. In one embodiment, the criticality status may further indicate one or more criticality levels assigned to a critical cache line.In one embodiment, the control circuit may be configured to update the replacement data based on a request encountered for a second given cache line in the cache. The replacement data may be updated to move a position of a second entry storing the second given cache line closer to a most recently accessed position based on the criticality value assigned to the second given cache line and the criticality values ​​of other cache lines represented in the replacement data.For example, in the event that the criticality value assigned to the second given cache line is lower than the criticality values ​​of one or more other cache lines represented in the replacement data, the control circuit may be configured to update the replacement data such that the second given cache line is represented at a second position below positions occupied by the one or more other cache lines.

[0071] In one embodiment, the control circuit is configured to monitor a cache hit rate for cache lines at a plurality of low positions in the replacement data and to indicate criticality values ​​with critical status. The control circuit may be configured to ignore the criticality values ​​for victim selection and replacement data update based on the cache hit rate being below a threshold. In one embodiment, the control circuit may be configured to monitor a snoop hit rate for snoops that cause cache lines to be forwarded from the cache. The control circuit may be configured to ignore the criticality values ​​for victim selection and replacement data update based on the snoop hit rate exceeding a threshold. computer system

[0072] Next, referring to Fig. 13, a block diagram of one embodiment of a system 700 is shown. In the illustrated embodiment, the system 700 includes at least one instance of a system on a chip (SOC) 706 coupled to one or more peripherals 704 and an external memory 702. A power supply unit (PMU) 708 is provided that supplies supply voltages to the SOC 706 and one or more supply voltages to the memory 702 and / or the peripherals 704. In some embodiments, more than one instance of the SOC may be included (and more than one memory 702 may also be included). The memory 702, in one embodiment, may have the Fig. 1 and Fig. 12. The SOC 706 may, in one embodiment, include an instance of the memory 18 illustrated in Fig. 12 illustrated SOC 300.

[0073] The peripherals 704 may include any desired circuitry depending on the type of system 700. For example, in one embodiment, the system 704 may be a mobile device (e.g., a personal digital assistant (PDA), a smartphone, etc.), and the peripherals 704 may include devices for various types of wireless communication, such as Wi-Fi, Bluetooth, cellular, global positioning system, etc. The peripherals 704 may also include additional storage, including RAM storage, solid-state storage, or disk storage. The peripherals 704 may include user interface devices, such as a display screen, including touch display screens or multi-touch display screens, keyboard or other input devices, microphones, speakers, etc.In other embodiments, system 700 may be any type of computing system (e.g., desktop personal computer, laptop, workstation, nettop, etc.).

[0074] The external memory 702 may include any type of memory. For example, the external memory 702 may be SRAM, dynamic RAM (DRAM), such as synchronous DRAM (SDRAM), double data rate SDRAM (DDR, DDR2, DDR3, etc.), RAMBUS DRAM, low-performance versions of DDR DRAM (e.g., LPDDR, mDDR, etc.), etc. The external memory 702 may include one or more memory modules to which the memory devices are attached, such as single in-line memory modules (SIMMs), dual in-line memory modules (DIMMs), etc. Alternatively, the external memory 702 may include one or more memory devices attached to the SOC 706 in a chip-on-chip or package-on-package implementation.

[0075] As illustrated, system 700 is shown to have application in a wide range of fields. For example, system 700 may be utilized as part of the chips, circuitry, components, etc., of a desktop computer 710, laptop computer 720, tablet computer 730, cellular or mobile phone 740, or television 750 (or a set-top box coupled to a television). Also illustrated are a smartwatch and a health monitoring device 760. In some embodiments, a smartwatch may include a variety of functions related to general computing. For example, a smartwatch may provide access to email, cellular service, a user calendar, and so on. In various embodiments, a health monitoring device may be a dedicated medical device or otherwise include dedicated health-related functionality.For example, a health monitoring device may monitor a user's vital signs, track a user's proximity to other users for the purpose of epidemiological distancing, track contact tracing, provide communication to emergency services in the event of a health emergency, and so on. In various embodiments, the aforementioned smartwatch may include some or no health monitoring-related functions. Other wearable devices are also contemplated, such as devices worn around the neck, devices implantable in the human body, eyewear designed to provide an augmented and / or virtual reality experience, and so on.

[0076] The system 700 may further be used as part of a cloud-based service(s) 770. For example, the aforementioned devices and / or other devices may access computing resources in the cloud (i.e., remotely located hardware and / or software resources). Still further, the system 700 may be utilized in one or more household devices other than those previously mentioned. For example, devices within the home may monitor and detect conditions requiring attention. For example, various devices within the home (e.g., a refrigerator, a cooling system, etc.) may monitor the status of the device and provide an alert to the homeowner (or, for example, a repair facility) should a particular event be detected.Alternatively, a thermostat can monitor the temperature in the apartment and can automate settings of a heating / cooling system based on a history of responses to various conditions by the apartment owner. Also in . Fig. 13 illustrates the application of system 700 in various modes of transportation. For example, system 700 may be used in the control and / or entertainment systems of airplanes, trains, buses, rental cars, passenger vehicles, watercraft, from private boats to cruise ships, (rental or private) scooters, and so on. In various cases, system 700 may be used to provide automated guidance (e.g., self-driving vehicles), general system control, and so on. Any of these many other embodiments are possible and are contemplated. It is noted that the Fig. The devices and applications illustrated in Figure 13 are illustrative only and are not intended to be limiting. Other devices are possible and are contemplated. Computer-readable storage medium

[0077] Now referring to Fig. 14, a block diagram of one embodiment of a computer-readable storage medium 800 is shown. Generally speaking, a computer-accessible storage medium may include any storage medium that can be accessed by a computer during use to provide instructions and / or data to the computer. For example, a computer-accessible storage medium may include storage media such as magnetic or optical media, e.g., disks (fixed or portable), tapes, CD-ROM, DVD-ROM, CD-R, CD-RW, DVD-R, DVD-RW, or Blu-ray. Storage media may further include volatile or non-volatile storage media, such as RAM (e.g., synchronous dynamic RAM (SDRAM), Rambus DRAM (RDRAM), static RAM (SRAM), etc.), ROM, or flash memory. The storage media may be physically enclosed within the computer to which the storage media provides instructions / data.Alternatively, the storage media may be connected to the computer. For example, the storage media may be connected to the computer via a network or wireless connection, such as network storage. The storage media may be connected via a peripheral interface, such as the Universal Serial Bus (USB). In general, the computer-accessible storage medium 800 may store data in a non-transitory manner, where non-transitory in this context may refer to the instructions / data not being transmitted on a signal. For example, the non-transitory storage may be volatile (losing the stored instructions / data in response to a shutdown), or it may be non-volatile.

[0078] The computer-accessible storage medium 800 in Fig. 14 may store a database 804 representative of the SOC 300. In general, the database 804 may be a database that can be read by a program and used directly or indirectly to manufacture the hardware comprising the SOC 300. For example, the database may be a behavioral-level or register-transfer-level (RTL) description of the hardware functionality in a high-level design language (HDL), such as Verilog or VHDL. The description may be read by a synthesis tool, which may synthesize the description to generate a netlist that includes a list of gates from a synthesis library. The netlist includes a set of gates that also represent the functionality of the hardware comprising the SOC 300. The netlist may then be placed and routed to generate a data set describing geometric shapes to be applied to masks.The masks can then be used in various semiconductor fabrication steps to produce a semiconductor circuit or circuits corresponding to the SOC 300. Alternatively, the database 804 on the computer-accessible storage medium 800 can be the netlist (with or without the synthesis library) or the dataset, as desired.

[0079] While the computer-accessible storage medium 800 stores a representation of the SOC 300, other embodiments may, as desired, store a representation of any portion of the SOC 100, including any subset of the Fig. 12. In addition, the database 804 may support the processors 10A-10N, the coprocessor 12, or both, as shown in Fig. 1, and may further represent the LLC 14 and / or the memory controller 16. The database 804 may represent any portion of the foregoing.

[0080] This disclosure includes references to "one" embodiment or groups of "embodiments" (e.g., "some embodiments" or "various embodiments"). Embodiments are various implementations or instances of the disclosed concepts. References to "embodiment," "an embodiment," "a particular embodiment," and the like do not necessarily refer to the same embodiment. A wide variety of possible embodiments are contemplated, including those specifically disclosed, as well as modifications or alternatives that fall within the spirit or scope of the disclosure.

[0081] This disclosure may discuss potential advantages that may arise from the disclosed embodiments. Not all implementations of these embodiments necessarily exhibit any or all of the potential advantages. Whether an advantage is achieved for a particular implementation depends on many factors, some of which are outside the scope of this disclosure. Indeed, there are a number of reasons why an implementation that falls within the scope of the claims may not exhibit some or all of the disclosed advantages. For example, a particular implementation might include different circuitry outside the scope of the disclosure that, when used in conjunction with one of the disclosed embodiments, negates or reduces one or more of the disclosed advantages. Furthermore, a suboptimal design of a particular implementation (e.g.,Implementation techniques or tools) may cancel out or diminish any benefits disclosed. Even assuming a qualified implementation, the achievement of benefits may still depend on other factors, such as the environmental circumstances in which the implementation is provided. For example, inputs provided to a particular implementation may prevent one or more problems addressed in this disclosure from occurring on a particular occasion, thereby possibly failing to achieve the benefit of its solution. Due to the existence of possible factors external to this disclosure, it is expressly intended that any potential benefits described herein should not be construed as claim limitations that must be met to prove infringement.Rather, the identification of such potential benefits is intended to illustrate the type(s) of improvement available to designers benefiting from this disclosure. The description of such benefits in a permissive sense (e.g., stating that a particular benefit "may occur") is not intended to cast doubt on whether such benefits can actually be achieved, but rather to acknowledge the technical reality that the achievement of such benefits often depends on additional factors.

[0082] Unless otherwise stated, embodiments are not restrictive. That is, the disclosed embodiments are not intended to limit the scope of any claims based on this disclosure, even if only a single example is described with respect to a particular feature. The disclosed embodiments are intended to be illustrative and not restrictive, unless otherwise stated in the disclosure. Thus, the application is intended to enable claims to cover disclosed embodiments, as well as the alternatives, modifications, and equivalents that would be obvious to one skilled in the art having the benefit of this disclosure.

[0083] For example, features in this application may be combined in any suitable manner. Accordingly, during the prosecution of this application (or an application claiming priority hereof), new claims may be drafted to any such combination of features. In particular, with reference to the appended claims, features from dependent claims may be combined with those of other dependent claims where appropriate, including claims that depend on other independent claims. Similarly, features from respective independent claims may be combined where appropriate.

[0084] While the appended dependent claims may be drafted such that each depends on a single other claim, additional dependencies are contemplated accordingly. All combinations of features in the dependent claims that are consistent with this disclosure are contemplated and may be claimed in this or another application. In summary, combinations are not limited to those specifically enumerated in the appended claims.

[0085] Where appropriate, it is also contemplated that claims formulated in one format or statutory type (e.g., facility) should support corresponding claims of a different format or statutory type (e.g., method).

[0086] Because this disclosure is a legal document, various terms and phrases may be subject to regulatory and legal interpretation. Notice is hereby given that the following paragraphs, as well as definitions provided throughout this disclosure, should be used in determining how claims based on this disclosure are to be interpreted.

[0087] References to a singular form of an element (i.e., a noun or noun phrase preceded by "a" or "the") are intended to mean "one or more" unless the context clearly dictates otherwise. Thus, a reference to "an element" in a claim, without an accompanying context, does not exclude additional instances of the element. A "plurality" of elements refers to a set of two or more of the elements.

[0088] The word “may” is used herein in a permissive sense (i.e., having the potential, being able to) and not in a mandatory sense (i.e., must).

[0089] The terms “comprehensive” and “including” and forms thereof are open-ended and mean “including, but not limited to.”

[0090] When the term "or" is used in this disclosure with reference to a list of options, it is generally understood to be used in the inclusive sense unless the context indicates otherwise. Thus, a statement of "x or y" is equivalent to "x or y or both," and thus covers 1) x but not y, 2) y but not x, and 3) both x and y. On the other hand, a phrase such as "either x or y, but not both" makes it clear that "or" is used in the exclusive sense.

[0091] A statement of "w, x, y, or z, or any combination thereof" or "at least one of ... w, x, y, and z" is intended to cover all possibilities involving a single element up to the total number of elements in the set. For example, for the set [w, x, y, z], these phrases cover any single element of the set (e.g., w, but not x, y, or z), any two elements (e.g., w and x, but not y or z), any three elements (e.g., w, x, and y, but not z), and all four elements. The phrase "at least one of ... x, y, and z" thus refers to at least one element of the set [w, x, y, z], covering all possible combinations in this list of elements. This phrase should not be interpreted to require that at least one instance of w, at least one instance of x, at least one instance of y, and at least one instance of z be present.

[0092] Various "labels" may precede nouns or noun phrases in this disclosure. Unless the context indicates otherwise, various labels used for a feature (e.g., "first circuit," "second circuit," "particular circuit," "given circuit," etc.) refer to different instances of the feature. Additionally, the labels "first," "second," and "third," when applied to a feature, do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) unless otherwise noted.

[0093] The phrase "based on" or "upon the basis of" is used to describe one or more factors that influence a determination. This term does not preclude the possibility that additional factors may influence the determination. That is, a determination may be based solely on stated factors or may be based on the stated factors as well as other, unstated factors. Consider the phrase "determine A based on B." This phrase specifies that B is a factor used to determine A or that influences the determination of A. This phrase does not preclude that the determination of A may also be based on another factor, such as C. This phrase is also intended to cover an embodiment in which A is determined solely based on B. As used herein, the phrase "based on" is synonymous with the phrase "based at least in part on."

[0094] The phrases "in response to" and "in reaction to" describe one or more factors that trigger an effect. This phrase does not exclude the possibility that additional factors may influence or otherwise trigger the effect, either jointly with the specified factors or independently of the specified factors. That is, an effect may occur solely in response to these factors, or it may occur in response to the specified factors as well as other, unspecified factors. Consider the phrase "doing A in response to B." This phrase specifies that B is a factor that triggers the performance of A or that triggers a particular result for A. This phrase does not exclude that doing A may also occur in response to another factor, such as C. Nor does this phrase exclude that doing A may occur in response to B and C together.This phrase is also intended to cover an embodiment in which A is performed solely based on B. As used herein, the phrase "based on" is synonymous with the phrase "based at least in part on." Similarly, the phrase "in response to" is synonymous with the phrase "at least in part in response to."

[0095] Within this disclosure, various entities (which may be variously referred to as "units," "circuits," other components, etc.) may be described or claimed as being "configured" to perform one or more tasks or operations. This phrase—[entity] configured to [perform one or more tasks]—is used herein to refer to a structure (i.e., something physical). In particular, this phrase is used to indicate that this structure is arranged to perform the one or more tasks during operation. A structure may be referred to as being "configured to" perform a task even if the structure is not currently operating.Thus, an entity described or specified as being “configured to” perform a task refers to something physical, such as a device, a circuit, a system including a processing unit and memory storing program instructions executable to implement the task, etc. This phrase is not used herein to refer to anything intangible.

[0096] In some cases, various units / circuits / components may be described herein as performing a set of tasks or operations. It is understood that these entities are "configured to" perform these tasks / operations, even if not specifically stated.

[0097] The term "configured to" is not intended to imply "configurable to." For example, an unprogrammed FPGA would not be considered "configured to" perform a specific function. However, this unprogrammed FPGA can be "configurable to" perform that function. After appropriate programming, the FPGA can then be said to be "configured to" perform that specific function.

[0098] For purposes of U.S. patent applications based on this disclosure, a claim that a structure is "configured to" perform one or more tasks expressly does not invoke the application of 35 U.S.C. § 112(f) for that claim element. If, during the prosecution of a U.S. patent application based on this disclosure, the applicant wishes to invoke the application of Section 112(f), the applicant shall specify claim elements using the construct "means for" [performing a function].

[0099] Various "circuits" may be described in this disclosure. These circuits, or "circuitry," constitute hardware that includes various types of circuit elements, such as combinational logic, clocked storage devices (e.g., flip-flops, registers, latches, etc.), finite state machines, memories (e.g., random access memory, embedded dynamic random access memory), programmable logic arrays, and so on. Circuitry may be user-defined or taken from standard libraries. In various implementations, circuitry may include digital components, analog components, or a combination of both, as appropriate. Certain types of circuitry may be referred to generically as "units" (e.g., a decoding unit, an arithmetic logic unit (ALU), a functional unit, a memory management unit (MMU), etc.).Such units also refer to circuits or circuit logic.

[0100] The disclosed circuits / units / components and other elements illustrated in the drawings and described herein thus include hardware elements such as those described in the preceding paragraph. In many cases, the internal arrangement of hardware elements within a particular circuit can be specified by describing the function of that circuit. For example, a particular "decode unit" may be described as performing the function of "processing an opcode of an instruction and redirecting that instruction to one or more of a plurality of functional units," meaning that the decode unit is "configured to" perform that function. This functional statement is sufficient for one skilled in the computer arts to further identify a set of possible structures for the circuit.

[0101] In various embodiments, as discussed in the preceding paragraph, circuits, units, and other elements are defined by the functions or operations they are configured to implement. The arrangement of these circuits / units / components with respect to one another and the manner in which they interact form a microarchitectural definition of the hardware that is ultimately fabricated on an integrated circuit or programmed into an FPGA to form a physical implementation of the microarchitectural definition. Thus, the microarchitectural definition will be recognized by those skilled in the art as a structure from which many physical implementations can be derived, all of which fall within the broader structure described by the microarchitectural definition.This means that one skilled in the art, presented with the microarchitecture definition provided in accordance with this disclosure, can, without undue experimentation and by exercising ordinary skill, implement the structure by coding the description of the circuits / units / components in a Hardware Description Language (HDL), such as Verilog or VHDL. The HDL description is often expressed in a manner that may appear functional. However, to one skilled in the art, this HDL description is the manner used to transform the structure of a circuit, unit, or component to the next level of implementation detail. Such an HDL description may take the form of behavioral code (which is typically not synthesizable), Register Transfer Language (RTL) code (which, unlike behavioral code, is typically synthesizable), or structural code (e.g.,A netlist (i.e., a netlist specifying logic gates and their connectivity) can be used. The HDL description can then be synthesized against a library of cells designed for a given integrated circuit manufacturing technology and can be modified for timing, power, and other reasons to result in a final design database that is submitted to a foundry to create masks and ultimately manufacture the integrated circuit. Some hardware circuits, or sections thereof, can also be customized in a schematic editor and incorporated into the integrated circuit design along with synthesized circuit logic. The integrated circuits can include transistors and other circuit elements (e.g., passive elements such as capacitors, resistors, inductors, etc.) and provide interconnection between the transistors and circuit elements.Some embodiments may implement multiple integrated circuits coupled together to implement the hardware circuits, and / or discrete elements may be used in some embodiments. Alternatively, the HDL design may be synthesized into a programmable logic array, such as a field-programmable gate array (FPGA), and may be implemented in the FPGA. This decoupling between the design of a group of circuits and the subsequent low-level implementation of those circuits typically leads to the scenario where the circuit or logic designer never specifies a specific set of structures for the low-level implementation beyond a description of what the circuit is configured to do, since that process is performed at a different stage of the circuit implementation process.

[0102] The fact that many different low-level combinations of circuit elements can be used to implement the same circuit specification results in a large number of equivalent structures for that circuit. As noted, these low-level circuits can vary according to changes in manufacturing technology, the foundry selected to manufacture the integrated circuit, the library of cells provided for a particular project, etc. In many cases, the selections made by different design tools or methodologies to produce these different implementations can be arbitrary.

[0103] Furthermore, it is common for a single implementation of a particular functional specification of a circuit to include a large number of devices (e.g., millions of transistors) for a given embodiment. Accordingly, the sheer volume of this information makes it impractical to provide a complete specification of the low-level structure used to implement a single embodiment, let alone the vast array of equivalent possible implementations. For this reason, the present disclosure describes a structure of circuits using the functional shorthand commonly employed in the industry.

[0104] Various embodiments are contemplated as set forth in the following numbered examples: 1. System comprising: one or more processors configured to issue memory requests to access a memory system; and a cache coupled to the one or more processors and configured to cache data from the memory system for access by the one or more processors, wherein: the cache comprises a control circuit configured to assign criticality values ​​to cache lines, wherein a given criticality value corresponds to a given cache line; the cache is configured to preferentially retain cache lines identified as critical based on the corresponding criticality values ​​over cache lines not identified as critical during selection of a victim cache line for replacement, the cache further configured to select the victim cache line according to replacement data maintained by the cache separate from the criticality values; the control circuit is configured to detect one or more indications that at least some of the cache lines identified as critical are no longer critical; and the control circuit is configured to terminate preferential retention of the cache lines based on the one or more indications. 2. The system of example 1, wherein the control circuit is configured to monitor a cache hit rate for cache lines located at a plurality of least recently used positions in the replacement data and having criticality values ​​indicating a critical status, and wherein one of the one or more indications is based on the cache hit rate being below a threshold. 3. The system of example 1 or 2, wherein the control circuit is configured to monitor a rate at which snoop hits occur in the cache and cause a forwarding of a corresponding cache line in response to the snoop hit, and wherein one of the one or more indications is based on the snoop hit rate being above a threshold. 4. The system of any of examples 1 to 3, further comprising a coprocessor coupled to the cache and configured to execute coprocessor instructions issued by the one or more processors to the coprocessor, and wherein the one or more indications comprise memory requests issued by the coprocessor to the cache. 5. The system of example 4, wherein the control circuit is configured to derive the coprocessor memory requests based on prefetch requests generated by the one or more processors, indicated as coprocessor prefetch requests. 6. The system of any preceding example, further comprising a second cache coupled to the cache, wherein the second cache is configured to provide an indication of the capacity in the second cache attributable to the data from the cache, and wherein the criticality values ​​indicate non-critical and a plurality of criticality levels, and wherein the control circuit is configured to override the plurality of criticality levels with a lowest of the plurality of criticality levels based on the indication that the capacity is less than a threshold. 7. The system of any preceding example, wherein the control circuit is configured to terminate the preferential retention at least in part by ignoring the criticality values ​​for the victim selection and the replacement data update. 8. System comprising: one or more processors configured to issue memory requests to access a memory system; and a cache coupled to the one or more processors and configured to cache data from a memory system for access by one or more processors, the cache comprising control circuitry, and wherein: the control circuit is configured to assign criticality values ​​to the cache lines, wherein a given criticality value corresponds to a given cache line; the cache is configured to preferentially select cache lines that have been identified as critical based on the corresponding criticality values ​​over cache lines that have not been identified as critical when selecting a victim cache line as a replacement, wherein the cache is further configured to select the victim cache line according to replacement data maintained by the cache separate from the criticality values; the control circuit is configured to detect one or more indications that at least some of the cache lines identified as critical are no longer critical; and the control circuit is configured to accelerate the eviction of the cache lines identified as critical based on the one or more indications. 9. The system of example 8, wherein the control circuit is configured to monitor a cache hit rate for cache lines located at a plurality of least recently used positions in the replacement data and having criticality values ​​indicating a critical status, and wherein one of the one or more indications is based on the cache hit rate being below a threshold. 10. The system of example 8 or 9, wherein the control circuit is configured to monitor a rate at which snoop hits occur in the cache and cause a forwarding of a corresponding cache line in response to the snoop hit, and wherein one of the one or more indications is based on the snoop hit rate being above a threshold. 11. The system of any of examples 8 to 10, wherein the one or more indications comprise memory requests issued by a coprocessor to the cache, the coprocessor coupled to the cache and configured to execute coprocessor instructions issued by the one or more processors to the coprocessor. 12. The system of example 11, wherein the control circuit is configured to derive the coprocessor memory requests based on prefetch requests generated by the one or more processors, indicated as coprocessor prefetch requests. 13. The system of any of examples 8 to 12, further comprising a second cache configured to provide an indication of capacity in the second cache attributable to data from the cache, and wherein the criticality values ​​indicate non-critical and a plurality of criticality levels, and wherein the control circuit is configured to override the plurality of criticality levels with a lowest of the plurality of criticality levels based on the indication of capacity being lower than a threshold. 14. The system of any of examples 8 to 13, wherein the control circuit is configured to accelerate the emission at least in part by ignoring the criticality values ​​for victim selection and replacement data update. 15. A method comprising: Assigning criticality values ​​to cache lines in a cache, where a given criticality value corresponds to a given cache line; during the selection of a victim cache line for replacement, preferentially retaining cache lines identified as critical based on the corresponding criticality values ​​over cache lines not identified as critical, wherein the selection is further based on replacement data based on values ​​that are kept separate from the criticality values ​​by the cache; Detecting one or more indications that at least some of the cache lines identified as critical are no longer critical; and Ignoring the criticality values ​​for victim selection and replacement data update based on the one or more indications. 16. The method of example 15, further comprising monitoring a cache hit rate for cache lines located at a plurality of least recently used locations in the replacement data and having criticality values ​​indicating a critical status, and wherein one of the one or more indications is based on the cache hit rate being below a threshold. 17. The method of example 15 or 16, further comprising monitoring a rate at which snoop hits occur in the cache and cause a forwarding of a corresponding cache line in response to the snoop hit, and wherein one of the one or more indications is based on the snoop hit rate being above a threshold. 18. The method of any of examples 15 to 17, wherein the one or more indications comprise memory requests issued by a coprocessor to the cache, the coprocessor coupled to the cache and configured to execute coprocessor instructions issued by one or more processors to the coprocessor. 19. The method of example 18, further comprising deriving the coprocessor memory requests based on prefetch requests generated by one or more processors, specified as coprocessor prefetch requests. 20. The method of any one of examples 15 to 19, further comprising: Providing an indication from a second cache of the capacity in the second cache that is attributable to data from the cache, wherein the criticality values ​​indicate non-critical and a plurality of criticality levels; and Overriding the plurality of criticality levels with a lowest of the plurality of criticality levels based on the capacity specification being lower than a threshold.

[0105] Numerous variations and modifications will become apparent to those skilled in the art once the above disclosure is fully understood. It is intended that the following claims be interpreted to encompass all such variations and modifications.

Claims

[1] System (700), comprising: one or more processors (10A-10N) configured to issue memory requests to access a memory system (702); and a cache (14) coupled to the one or more processors (10A-10N) and configured to cache data from the memory system (702) for access by the one or more processors (10A-10N), wherein: the cache (14) comprises a control circuit (34) configured to assign criticality values ​​to cache lines based on a plurality of factors at a time the cache lines are filled into the cache (14); the control circuit (34) is configured to, during the fill of a given cache line, represent the given cache line at a selected position in replacement data for the cache (14) based on the criticality value assigned to the given cache line; the control circuit (34) is configured to select a victim cache line to be evicted from the cache (14) based on the replacement data; and the control circuit (34) is configured to determine, according to a probability-based mode selection for a given cache eviction operation, whether to proceed according to a first victim selection mode or a second victim selection mode, wherein the first victim selection mode prevents the selection of cache lines with a critical status as victims and the second victim selection mode allows the selection of a cache line with a critical status as victims. [2] The system (700) of claim 1, further comprising a second cache (28) coupled to the cache (14) and configured to cache data from the memory system (702) for the cache (14) and for one or more other cache access agents in the system (700), the second cache (28) configured to store the victim cache line and maintain an indication of the criticality value assigned to the victim cache line by the control circuit (34). [3] The system (700) of claim 2, further comprising a memory controller (16) configured to control one or more memory devices forming at least a portion of the memory system (702), and wherein the memory controller (16) includes the second cache (28). [4] The system (700) of claim 2, wherein the second cache (28) is configured to provide the criticality value to the cache (14) with the victim cache line in a fill based on another memory request made after eviction of the victim cache line from the cache (14). [5] The system (700) of claim 2, wherein the second cache (28) is configured to maintain second replacement data, and wherein the second cache (28) is configured to evict cache lines from the second cache based on the second replacement data, and wherein an initial position of the victim cache line in the second replacement data is based on the criticality value. [6] The system (700) of claim 5, further comprising a monitor circuit coupled to the second cache (28) and configured to provide an indication of the capacity in the second cache (28) attributable to data from the cache (14), and wherein the cache (14) is configured to generate an insertion hint to be transmitted with the victim cache line based on the indication of the capacity. [7] The system (700) of claim 6, wherein the cache (14) comprises a table coupled to the control circuit (34) that maps ranges of the capacity indication to values ​​for the insertion hint. [8] The system (700) of claim 6, wherein the second cache (28) includes a second table, and wherein the second cache (28) is configured to select an entry in the second table based on the insertion hint, and wherein the second table is configured to output an insertion point indication from the selected entry. [9] The system (700) of claim 1, wherein the criticality values ​​include critical and non-critical values. [10] The system (700) of claim 9, wherein the criticality values ​​further indicate one of a plurality of criticality levels assigned to a critical cache line. [11] The system (700) of claim 1, wherein the control circuit (34) is configured to update the replacement data based on a request encountered for a second given cache line in the cache (14), and wherein the replacement data is updated to move a position of a second entry storing the second given cache line closer to a most recently accessed position based on the criticality value assigned to the second given cache line and the criticality values ​​of other cache lines represented in the replacement data. [12] The system (700) of claim 11, wherein, in the event that the criticality value assigned to the second given cache line is lower than the criticality values ​​of one or more other cache lines represented in the replacement data, the control circuit (34) is configured to update the replacement data to represent the second given cache line at a second position among positions occupied by the one or more other cache lines. [13] The system (700) of claim 1, wherein the control circuit (34) is configured to monitor a cache hit rate for cache lines located at a plurality of low positions in the replacement data and having criticality values ​​indicating a critical status, and wherein the control circuit (34) is configured to ignore the criticality values ​​for victim selection and replacement data update based on the cache hit rate being below a threshold. [14] The system (700) of claim 1, wherein the control circuit (34) is configured to monitor a snoop hit rate for snoops that cause cache lines to be forwarded from the cache (14), and wherein the control circuit (34) is configured to ignore the criticality values ​​for victim selection and replacement data update based on the snoop hit rate exceeding a threshold. [15] Method comprising: Assigning criticality values ​​to cache lines in a cache (14) based on a plurality of factors at a time the cache lines are filled into the cache (14); Representing a given cache line at a selected position in replacement data for the cache (14) during the fill of the given cache line based on the criticality value assigned to the given cache line; and Selecting a victim cache line to be evicted from the cache (14) based on the replacement data, including determining, according to a probability-based mode selection for a given cache eviction operation, whether according to a first victim selection mode or a second victim selection mode is to be used, wherein the first victim selection mode prevents the selection of cache lines with a critical status as victims and the second victim selection mode allows the selection of a cache line with a critical status as victims. [16] The method of claim 15, further comprising: Caching data from a memory system (702) for the cache (14) in a second cache coupled to the cache (14), including storing the victim cache line; Maintaining an indication of the criticality value assigned to the victim cache line by the cache (14); and Providing the criticality value with the victim cache line in a fill to the cache (14) based on a further memory request made after eviction of the victim cache line from the cache (14). [17] The method of claim 16, further comprising: Maintaining second replacement data in the second cache (28), wherein the second cache (28) is configured to evict cache lines from the second cache (28) based on the second replacement data; and Insert the victim cache line at an initial position in the second replacement data based on the criticality value. [18] The method of claim 15, wherein the criticality values ​​include critical and non-critical values. [19] The method of claim 18, wherein the criticality values ​​further indicate one of a plurality of criticality levels assigned to a critical cache line. [20] A cache (14) configured to cache data from a memory system (702) for access by one or more processors (10A-10N), the cache (14) comprising a control circuit (34), wherein: the control circuit (34) is configured to assign criticality values ​​to cache lines based on a plurality of factors at a time the cache lines are filled into the cache (14); the control circuit (34) is configured to, during the fill of a given cache line, represent the given cache line at a selected position in replacement data for the cache (14) based on the criticality value assigned to the given cache line; the control circuit (34) is configured to select a victim cache line to be evicted from the cache (14) based on the replacement data; and the control circuit (34) is configured to determine, according to a probability-based mode selection for a given cache eviction operation, whether to proceed according to a first victim selection mode or a second victim selection mode, wherein the first victim selection mode prevents the selection of cache lines with a critical status as victims and the second victim selection mode allows the selection of a cache line with a critical status as victims.

Images