Caching guidelines informing about criticality

Abstract

System comprising: one or more processors (10A - 10N) configured to issue memory requests for accessing a memory system (18); and a cache (32) coupled to the one or more processors (10A - 10N) and configured to temporarily store data from the memory system (18) for access by the one or more processors (10A - 10N), wherein: the cache (32) includes a control circuit (34) configured to assign criticality values ​​to cache rows, a given criticality value corresponding to a given cache row;the cache (32) is configured to preferentially retain cache rows identified as critical based on their criticality values ​​over cache rows not identified as critical when selecting a sacrifice cache row as a replacement, wherein the cache (32) is further configured to select the sacrifice cache row according to replacement data kept separate from the criticality values ​​by the cache (32); the cache (32) is configured to monitor a cache hit rate for cache rows with criticality values ​​indicating a critical status; the control circuit (34) is configured to detect one or more indications that at least some cache rows identified as critical are no longer critical, one or more indications being based on the cache hit rate being below a threshold;and the control circuit (34) is configured to stop preferentially retaining the cache lines based on one or more specifications.;

Description

BACKGROUNDTechnical FieldThe embodiments described herein relate to caches in computer systems, and more particularly, to caching policies.Description of the Prior ArtCaches have long been used in digital systems to reduce effective memory latency by capturing a copy of data accessed by a processor, coprocessor, or other digital device in a cache memory that is locally on the device. The cache memory may be smaller than the main storage system and may be optimized for low latency (whereas the main storage system is often optimized for storage density, which is at the expense of latency). Accordingly, the cache memory itself may reduce latency. In addition, cache memory may be local to the device, and thus latency may be reduced because there is no transport delay to the memory controller / main storage system and back to the device. Moreover, the cache may be private to the device or a small number of devices (e.g., processor / coprocessor clusters), and thus contention for bandwidth to the cache may be reduced as compared to main memory.While caches reduce effective memory latency, they represent limited storage and are therefore prone to misses (which causes a fill from memory into the cache to obtain the data, in addition to providing the data to the requesting device when the miss concerns a read request or performing the update when the miss concerns a write request). The fill is allocated memory in the cache (e.g., a cache line or a cache block). The mapping may cause other data in the cache to need replacement (also referred to as emitting a cache line from the cache). There are a plurality of replacement policies for selecting the cache line to be emitted based on the cache geometry. For example, set associative caches have memory arranged as a two-dimensional array of cache lines: a "row" is selected based on a subset of the memory address of the cache line (referred to as a set), and the row includes a plurality of cache lines that are the "columns" of the array (referred to as paths). When a cache miss is detected and a fill is initiated, one of the paths is assigned to the fill. A popular replacement policy for set associative caches is the least recently used policy (LRU). With LRU, accesses to the cache lines in a set from the most recently invoked (most recently used, most recently used or MRU) to the least recently invoked (least recently used or LRU) are tracked. Typically, a cache line when accessed is updated to the MRU and the cache lines between the previous rank of the cache line and the previous MRU are adjusted. The LRU cache line may be selected for replacement when a cache miss occurs.US 2011 / 0 145 506 A1 discloses a cache memory whose cache lines store a weight that indicates the importance of the data based on coherency state and frequency of use.US 2021 / 0 406 170 A1 discloses a processor with cache and flash controller connected via a network to flash memory for reading or writing data.US 2018 / 0 004 661 A1 discloses a device with two caches, wherein the second cache manages the data state and independently allocates data in the event of a lack of space.US 2021 / 0 390 053 A1 discloses a host assisted prefetcher in which the host provides the prefetch configuration to reduce the resource overhead in the storage device.BRIEF DESCRIPTION OF THE DRAWINGSThe following detailed description will be made with reference to the accompanying drawings, which will now be briefly described. FIG. 1 is a block diagram of an embodiment of a portion of a system. FIG. 2 is a flow chart illustrating criticality determination for an embodiment. FIG. 3 is a table illustrating LRU insertion and update in an embodiment of a last level cache (LLC) shown in FIG. 1. FIG. 4 is a flow diagram illustrating an embodiment of victim selection in the LLC. FIG. 5 is a flowchart illustrating criticality determination for another embodiment. FIG. 6 is a flow diagram illustrating LRU insertion in the LLC for one embodiment. FIG. 7 is a flow chart illustrating LRU stepping up in the LLC for one embodiment. FIG. 8 is a flow diagram illustrating victim selection for an embodiment. FIG. 9 is a flow diagram illustrating victim selection for another embodiment. FIG. 10 is a flow chart illustrating an assertion of an assertion for cache lines marked as critical. FIG. 11 is a flow diagram illustrating LRU insertion for cache lines in the memory cache for one embodiment. FIG. 12 is a block diagram of an embodiment of a system on a chip (SOC). FIG. 13 is a block diagram of various embodiments of a computer system. Figure 14 is a block diagram of one embodiment of a computer-accessible storage medium.While embodiments described in this disclosure are susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will be described in detail herein. It should be understood, however, that the drawings and detailed description thereto are not intended to limit the embodiments of the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the appended claims. The headings used herein are for organization purposes only and are not intended to be used to limit the scope of the description.DETAILED DESCRIPTION OF EMBODIMENTSThe invention is set forth in the appended claims.While the LRU replacement policy often provides good performance (e.g., cache hit rates remain high and thus memory latency is effectively reduced), there are cases where performance may be limited. For example, if the contention for cache lines is high and thus there are frequent omissions, some cache lines may be emitted, which when accessed again cause a higher loss of performance of the requesting device than other cache lines. For example, if a number of operations in the requesting device depend on the data in the cache line directly or indirectly through other operations, the requesting device may stall while waiting for the data. Other cache lines with fewer dependencies may be less critical to performance. The LRU policy has no way to reflect the differences in the criticality of cache lines.In one embodiment, a system comprising one or more processors and a cache coupled to the one or more processors may categorize cache lines according to one or more criticality levels based on one or more criteria measured at the time of filling the cache lines into the cache. The criteria may be selected to attempt to identify the cache lines that, when representing a cache miss, have a greater impact on the performance of the processors than other cache lines. Each cache line may have a criticality value that indicates its criticality level. For example, the critical value may indicate a non-critical status or a critical status. In one embodiment, the critical status may also include multiple criticality levels, as described in more detail below. In another embodiment, the critical status may be a single level that indicates critically, as opposed to the non-critical status.The cache may implement a replacement policy that uses the criticality values of the cache lines as a factor. For example, an LRU policy may be used, but the policy may be modified to account for the criticality of different cache lines. Cache lines having a criticality value indicative of a critical status ("critical cache lines") may be inserted into the LRU replacement data at the MRU location, while cache lines having criticality values indicative of a non-critical status ("non-critical cache lines") may be inserted into the data at lower locations (e.g., closer to the LRU location). In one embodiment, criticality values may also affect the update of the LRU replacement data. While LRU is used as an example replacement policy, other embodiments may implement other replacement policies. For example, a variety of pseudo-LRU policies may be used that approximate the LRU operation by making simplifications to make the policy easier to implement, particularly in large set associative caches. In addition, random replacement policies may be used and criticality may be used to reduce the likelihood that critical lines are selected. Policies based on least used may be used and critical rows may be selectively maintained in a similar manner as described below for LRU. Policies based on last in, first out, or first in, first out may be used, and critical cache lines may be at least partially excluded from LIFO or FIFO replacement. Each of these policies may be modified to account for criticality.In one embodiment, the system may include one or more additional cache stages between the aforementioned cache and system memory. For example, a memory cache may be used that is implemented on the memory controller that controls system memory. The criticality values of cache lines may be swapped between the caches when the cache lines are dispatched and invoked again, maintaining the criticality values while the cache lines remain cached in the cache hierarchy. Once the cache line has been removed from the cache hierarchy (and thus the data is only present in system memory), the criticality value may be lost.FIG. 1 is a block diagram of an embodiment of a system that includes a plurality of processors 10A- 10N, a coprocessor 12, a last level cache (LLC) 14, a memory controller 16, and a memory 18. Processors 10A- 10N and coprocessor 12 are coupled to LLC 14, which is coupled to memory controller 16, which is further coupled to memory 18. Processor 10N is illustrated in more detail, and other processors such as processor 10A may be similar. The processor 10N may include an instruction cache (ICache) 20, an instruction cache miss queue (IC miss queue) 22, an execution core 24 including a load queue (LDQ) 26, a data cache (DCache) 28, and a memory management unit (MMU) 30. The LLC 14 may include a cache 32, a criticality control circuit 34, and a memory cache insert lookup table (MCache insert LUT) 36. The memory cache 16 may include an insertion control circuit and LUT 38, an MCache 40, and a monitoring circuit 42.The ICache 20 may store instructions that are fetched by the processor 10N for execution by the execution core 24. If a fetch fails in the ICache 20, the fetch of the instruction cache line may be placed in the IC miss queue 22 and transmitted to the LLC 14 as a fill request for the ICache 20. Instructions executed by the execution core 24 may include load instructions (in short, loads). The loads may attempt to read data from the DCache 28 and, if a load fails in the DCache 28, may be transmitted to the LLC 14 as a fill request for the DCache 28. The loads transmitted to the LLC 14 may remain in the LDQ 26 to wait for data.The MMU 30 may provide address translations for instruction fetch addresses and load / store addresses, including translation buffers (TLBs), which may be local to the ICache 20 and the execution core 24. The MMU 30 may optionally include one or more Stage 2 TLBs (L2 TLBs) as well as table pass circuitry to perform the translation table reads to obtain a translation for an address that causes misses in the TLBs. The MMU 30 may transmit the table pass reads to the LLC 14. In one embodiment, the MMU 30 may access the table-pass reads for a potential cache hit, DCache 28, before being transmitted to the LLC 14, and may not transmit the reads to the LLC 14 when hitting the DCache 28. In other embodiments, the page table data is not cached in the DCache 28, and the MMU 30 may transmit table-pass reads to the LLC 14.The LLC 14 includes the cache 32, which may have any capacity and configuration. Memory requests from the processors 10A- 10N and the coprocessor 12 may be checked for a hit in the cache 32, and the data may be returned as a fill to the ICache 20, the DCache 28, or the MMU 30 in the event of a hit. If a miss in cache 32 results from the memory request, LLC 14 may transmit a memory request to memory controller 16 and may return the fill to requesting processor 10A- 10N or coprocessor 12 in response to memory controller 16 returning a fill to LLC 14. The LLC 14 may also fill the data into the cache 32 in the event of a miss. Generally, "data" is used herein in the general sense to refer to both instructions fetched by the processors 10A-10N for execution and data read / written by the processors due to execution of the instructions (e.g., operand data and result data), particularly when dealing with cache lines of data.Additionally, at the time of fill, the LLC 14 may assign a criticality value to the cache line processor 10A- 10N / coprocessor 12. The criticality control circuit 34 may determine the criticality value and may update the cache 32 with the criticality value. For example, the cache tags in the cache 32 may include a criticality value field. The critical value may indicate a non-critical status or a critical status. As mentioned above, in some embodiments, there may be more than one critical status level. The criticality control circuit 34 may also determine the critical state level.The criticality control circuit 34 may take into account a variety of factors in assigning the criticality values to cache lines. For example, the criticality control circuit 34 is coupled to the MMU 30, IC miss queue 22, and LDQ 26. In particular, filters that are for table pass requests may be categorized as critical. A TLB miss will likely affect additional instruction fetches or load / store requests because a translation affects a rather large amount of data and code sequences tend to access data that is proximate to other recently fetched data. For example, a page may be 4 kilobytes in size, 16 kilobytes in size, or even larger, such as 1 megabyte or 2 megabyte. Any page size may be used. In addition, a load that is at the head of the LDQ 26 when the fill is made for the load may be the oldest outstanding load in the processor 10N. Thus, the load is likely to block the departure of other completed instructions or to block a number of instructions (either directly or indirectly) due to dependency on the load data. The filters for loads located at the head of the LDQ 25 may be assigned a critical status. Similarly, if there is a fill for an instruction fetch request and it is the oldest fetch request in the IC miss queue 22 (e.g., located at the head of the IC miss queue 22), then the fetching of the instructions is likely to be blocked by waiting for instructions. Such instruction fetches may be assigned a critical status. Other embodiments may include additional factors within a given processor 10A-10N, or subsets of the above factors and other factors, as desired. In one embodiment, a critical status may also be assigned to the requests from coprocessor 12. For example, an embodiment of coprocessor 12 may not include a cache, and thus LLC 14 is the first stage of cache available to coprocessor 12. Cache lines that have not been assigned a critical status may be assigned a non-critical status.In one embodiment, the criticality values assigned to cache lines may be maintained while the cache lines remain valid in the cache hierarchy. The criticality value is assigned by the criticality control circuit 34 and then propagated with the cache line as it is emitted from the cache 32 and transferred to the memory controller 16 where it may be cached in the MCache 40. When the dispatched cache line is placed in the MCache 40 after being dispatched from the cache 32, the criticality value may be maintained. If the dispatched cache line is not placed in the MCache 40 after being dispatched from the cache 32, the memory controller 16 may drop the criticality value and write the data to the memory 18. There may be a variety of factors that affect whether or not an emitted cache line is cached in the MCache 40. The MCache 40 is shared with other components of the system, and the MCache 40 may have quotas of how much data from a given component may be cached. If the LLC 14 has crossed the quota, the issued cache line may not be cached. Alternatively, the dispatched cache line may be cached and another LLC cache line cached in the MCache 40 may be dispatched.Subsequently, when a cache line previously cached by the LLC 14 is again accessed by the LLC 14, MCache 40 may provide the cache line to the cache 32 as a fill, and the criticality value previously associated with the cache line may also be provided. The criticality control circuit 34 may assign the previous criticality value provided to the cache line from the MCache 40 unless other factors from the processor 10A-10N that generated re-access to the cache line indicate an update to a critical state or to a higher level of the critical state. For example, a non-critical cache line from the non-critical state MCache 40 may be filled into the LLC 14 unless it is assigned the critical state at the time of the re-access fill (e.g., if the fill is for a load at the head of the LDQ 26, an instruction fetch at the head of the IC miss queue 22, or an MMU table pass request). A critical cache line from the MCache 40 may be filled as critical. In embodiments implementing multiple criticality status levels, a higher critical status level may be assigned to a critical cache line from the MCache 40, which is also currently indicated as critical by the above factors (LDQ 26 head, IC miss queue 22 head, or MMU request) by the criticality control circuit 34.In one embodiment, cache lines emitted from the LLC 14 may be cached in the MCache 40, and may be inserted at a selected location into the replacement data of the affected set of the MCache 40. If the issued cache line is a critical cache line, it may be inserted at the MRU location. If the emitted cache line is a non-critical cache line, it may be inserted at a position lower than the MRU (closer to the LRU). In one embodiment, the insertion point for non-critical cache lines may be dynamic. For example, the insertion point may be based on the amount of cache capacity in the MCache 40 occupied by cache lines of the LLC 14. The memory controller 16 may include a monitoring circuit 42 that monitors the capacitance of the MCache 40 associated with the CPU and provides the information ("CPU capacitance") to the criticality control circuit 34. The criticality control circuit 34 may use the CPU capacity value as an index to the MCache insert LUT 36 and may read an insert indication from the indexed entry when an emitted cache line is transferred to the memory controller 16. The insertion hint may be used as an index to an LUT 38 in the memory controller 16, and the associated insertion control logic may adjust the insertion point (e.g., when a portion of the cache is shut down, the insertion point should be within the currently used LRU positions). The MCache 40 may insert the emitted cache block at the insertion point.Accordingly, cooperative look-up tables may be used in this embodiment to determine the point of insertion for emitted cache lines in the non-critical cache line MCache 40. The LUTs may be programmable so that the software can optimize performance as desired.The CPU capacitance value can be measured in any desired manner. In one embodiment, the CPU capacity may indicate the average number of MCache paths occupied by cache lines from the LLC 14. In another embodiment, an approximate percentage of cache capacity may be provided.As mentioned previously, cache 32 may include a field (e.g., in the cache tag) for the criticality value. Similarly, MCache 40 may include a field in the cache tag for the criticality value. In another embodiment, the MCache 40 may include a record identifier (DSID) for each cache line that identifies matching cache lines according to one or more criteria. In general, cache blocks with the same DSID may originate from the same source component (e.g., LLC 14 or another component of the system, such as a peripheral component not shown in FIG. 1 ). The DSID may be stored in a field in the tag. The DSID may be used to distinguish non-critical and critical cache lines (e.g., by using one non-critical cache line DSID and another critical cache line DSID or multiple critical status levels DSIDs in embodiments that use multiple critical status levels). The MCache 40 may decode the DSID to determine the criticality value to be transmitted to the LLC 14 when providing a fill.In one embodiment, processors 10A- 10N may serve as a central processing unit (CPU) of the system. The CPU of the system includes the one or more processors executing the main control software of the system, such as an operating system. Generally, software executed by the CPU during use may control the other components of the system to realize the desired functionality of the system. Processors 10A-10N may also execute other software, such as application programs. The application programs may provide user functionality and may rely on the operating system with respect to lower level device control, scheduling, storage management, etc. Accordingly, processors 10A- 10N may also be referred to as application processors.Generally, a processor may include any circuitry and or microcode configured to execute instructions defined in an instruction set architecture implemented by the processor. Processors may include processor cores implemented with components other than system on a chip (SOC) on an integrated circuit or at other levels of integration. Processors may further include discrete microprocessors, as well as processor cores and / or microprocessors integrated into multichip module implementations; processors implemented as multiple integrated circuits; etc.In one embodiment, coprocessor 12 may be configured to accelerate certain operations. For example, an embodiment is contemplated wherein a coprocessor performs large scale matrix and vector manipulations (multiple operations per instruction). Coprocessor 12 may receive instructions transmitted from processors 10A-10N. That is, the instructions ("coprocessor instructions") executed by coprocessor 12 and the instructions ("processor instructions") executed by processors 10A-10N may be part of the same instruction set architecture and may be merged into a code sequence retrieved by the processor. Processor 10A- 10N may decode the instructions and identify the coprocessor instructions for transmission to coprocessor 12 and execute processor instructions. The coprocessor 12 may receive the coprocessor instructions from the processor 10A-10N, decode the coprocessor instructions, and execute the coprocessor instructions. The coprocessor instructions may include load / store instructions to read storage data for operands and write result data to memory (in one embodiment, both may be performed in LLC 14).It is noted that the number and type of various components of the system in FIG. 1 may vary from embodiment to embodiment. For example, there may be any number of processors 10A- 10N. There may be more than one coprocessor 12, and if multiple coprocessors are included, there may be multiple instances of the same coprocessor and / or different types of coprocessors. There may be more than one memory controller 16 and if multiple memory controllers are included, memory space may be distributed across the memory controllers.It should be appreciated that various instructions, memory requests, etc. are referred to above as more recent or older than other instructions, requests, etc. A given operation may be more recent than another operation if the given operation is derived from an instruction that is later in time than the instruction from which the other operation is derived in program order. Similarly, a given operation is older than another operation if the given operation is derived from an instruction that is prior to the instruction from which the other operation is derived in program order.FIGS. 2, 3-4 illustrate an embodiment in which criticality values are either a critical or a non-critical status. FIGS. 5, 6, 7, 8-9 illustrate an embodiment in which the critical status has more than one criticality level. FIG. 10 illustrates a mechanism for accelerating the removal of critical cache lines from the LLC 14 for an embodiment that may apply to both types of criticality values. FIG. 11 is a flow chart illustrating victim selection from the LLC 14 based on the accelerator mechanism of FIG. 10.Referring to FIG. 2, a flowchart is shown illustrating an embodiment of the criticality control circuit 34 to assign a criticality value to a cache line being filled into the LLC 14. Although the blocks are shown in a particular order for ease of understanding, other orders may be used. Blocks may be performed in parallel in combinatorial logic in the criticality control circuit 34. Blocks, combinations of blocks, and / or the flow pattern as a whole may be routed over multiple clock cycles. The criticality control circuit 34 may be configured to implement the operation shown in FIG. 2.If the fill is a cache line for an MMU table pass request (decision block 50, "yes" branch), the criticality control circuit 34 may assign a critical status for the criticality value associated with the cache line (block 52). If the fill is a load cache line that is at the head of the LDQ 26 (decision block 54, "yes" branch), the criticality control circuit 34 may assign a critical status for the criticality value associated with the cache line (block 52). If the fill is a cache line for an instruction cache miss that is at the head of the IC miss queue 22 (decision block 56, "yes" branch), the criticality control circuit 34 may assign a critical status for the criticality value associated with the cache line (block 52). If the fill is a cache line that has a critical status in the MCache (decision block 58, "yes" branch), the criticality control circuit 34 may assign a critical status for the criticality value associated with the cache line (block 52). If none of the above criteria apply (decision blocks 50, 54, 56, 58 and 60, "no" branches), the criticality control circuit 34 may assign a non-critical status for the criticality value associated with the cache line. In one embodiment, coprocessor requests from coprocessor 12 may also be assigned a critical status. In another embodiment, the coprocessor requests may be assigned a non-critical status.FIG. 3 is a table 62 illustrating the operation of an embodiment of the criticality control circuit 34 for updating the replacement data for a set based on a fill of a cache line into the cache 32 (insertion section 64) and based on a cache hit for a processor request from a processor 10A-10N (update section 66). The update of the replacement data may be based on the request type, the previous state of the cache block, and the criticality value. The LRU column of the table indicates the position in the LRU rank (from MRU to LRU) of the cache line being filled (in the inserting section 64) or the cache line being made by a request (in the updating section 66). Other cache lines in the set may be updated to reflect the change. For example, if the filled / hit cache line becomes MRU, the position of any other cache line may be moved from the current MRU to the previous position of the filled / hit cache line by one position towards LRU. When the filled / hit cache line in the replacement data is moved to a position other than MRU, each cache line having a position different from the current position of the filled / hit cache line may be moved toward LRU.In the inserting section 64, the previous state is zero because the cache line is filled into the cache 32. For this section, request types other than non-temporal (NT) require the replacement data to be updated to fill MRU for critical cache lines. If the fill concerns a prefetch request (data or instruction) and the criticality value is a non-critical status, the fill is made at LRU position N, which is near the LRU position, but is not the LRU position itself. For example, N may be about 25% of the distance between the LRU and the MRU above the LRU. For example, if cache 32 has 8 ways, 25% above the LRU would be 2 locations above the LRU. If cache 32 has 16 ways, 25% above the LRU would be 4 positions above the LRU. If the fill for a demand fetch (instruction or data) and the criticality value is a non-critical status, the fill is made at LRU position L (near the center of the replacement data area). For example, if cache 32 has 8 ways, in various embodiments, L may range from positions 4 to 6, assuming that the LRU position is numbered 0. If the cache has 16 ways, L may be in the range of 6 to 8. If the fill is for an NT demand call, the LRU position of the fill may be at position M, near the LRU, but less than N.In the embodiment of FIG. 3, the update of the replacement data to a hit of the cache lines may be independent of the criticality value of the cache line. Other embodiments may take into account criticality in the update. If the hit request is a demand fetch (instruction or data) and the cache line was a pre-fetched cache line, the LRU position may be unchanged (NC), but the pre-fetch trace bit may not be set for the cache line, so that it is a demand fetch the next time the cache line is hit. If the hit request is a demand fetch (instruction or data) and the hit cache line was an NT demand or a demand fetch (instruction or data), the hit cache line may be made MRU. If the hit request is a data prefetch, the hit cache line may be placed at N (near the LRU). If the hit request is an instruction prefetch, the hit cache line is made the MRU. If the hit request is an NT demand, the hit cache may be position N.Referring to FIG. 4, a flowchart is shown illustrating the operation of one embodiment of the criticality control circuit 34 for selecting a victim cache line to be emitted when a cache miss is detected. Although the blocks are shown in a particular order for ease of understanding, other orders may be used. Blocks may be performed in parallel in combinatorial logic in the criticality control circuit 34. Blocks, combinations of blocks, and / or the flow pattern as a whole may be routed over multiple clock cycles. The criticality control circuit 34 may be configured to implement the operation shown in FIG. 4.If at least one invalid cache entry exists in the set indexed by the cache miss (decision block 70, "yes" branch), the criticality control circuit 34 may select the LRU next invalid entry (block 72). An invalid entry may be a cache line storage location (e.g., way) that is not currently storing a cache line. The LRU-next invalid entry may be the invalid entry that is invalid and has a position closest to the LRU position in the replacement data compared to the positions of the other invalid entries. The LRU-next invalid entry may be in the LRU position.If there are no invalid entries in the set (decision block 70, "no" branch), the criticality control circuit 34 may select a valid entry as a victim. In a typical LRU policy, the LRU entry may be selected. However, in this embodiment, the criticality control circuit 34 may maintain the critical cache lines with some probability. Accordingly, corrupted pseudo random selection may be generated (e.g., based on a linear feedback shift register or LFSR and the desired probability) (block 74). Based on the pseudo random selection, the criticality control circuit 34 can selectively mask the critical cache lines against the selection (block 76). For example, if the corrupted pseudo random selection indicates an evaluation of the corrupted test (e.g., "yes"), the critical cache lines may not be masked. If the corrupted pseudo-random value indicates another evaluation of the corrupted test (e.g., "no"), the critical cache lines may be masked. The criticality control circuit 34 may select the LRU-next valid, non-masked entry and may emit the cache block in that entry (block 78).FIG. 5 is a flow diagram illustrating the operation of the criticality control circuit 34 for another embodiment of allocating a criticality for a cache line filled into the LLC 14. Although the blocks are shown in a particular order for ease of understanding, other orders may be used. Blocks may be performed in parallel in combinatorial logic in the criticality control circuit 34. Blocks, combinations of blocks, and / or the flow pattern as a whole may be routed over multiple clock cycles. The criticality control circuit 34 may be configured to implement the operation shown in FIG. 5.Similar to the embodiment of FIG. 2, the cache line may be critical when the cache line is filled as a result of an MMU table pass request (decision block 80, "yes" branch), a load at the head of the LDQ 26 (decision block 82, "yes" branch), or an instruction fetch at the head of the IC miss queue 22 (decision block 84, "yes" branch). In this embodiment, there are multiple levels of critical status. If the criticality provided by the MCache 40 indicates the critical status (decision block 86, "yes" branch), the criticality control circuit 34 may increase the level of critical status from the status provided by the MCache 40 (block 88). If the MCache 40 is indicated as not critical (decision block 86, "no" branch), either the cache line was not critical previously or the cache line was a miss in the MCache 40. In these cases, the criticality control circuit 34 may initialize the criticality value at the lowest level of the critical status (block 90).If the cache line in the current fill is not critical (decision blocks 80, 82, and 84, "no" branches), but the criticality value provided by the MCache 40 is a critical status (decision block 92, "yes" branch), the criticality control circuit 34 may maintain the criticality value provided by the MCache 40 (block 94). Otherwise (decision block 92, "no" branch), the criticality control circuit 34 may initialize the criticality value with non-critical status (block 96).FIG. 6 is a flow diagram illustrating the operation of an embodiment of the criticality control circuit 34 to update the replacement data for a set based on a fill of a cache line into the cache 32 (an insertion of a cache line). Although the blocks are shown in a particular order for ease of understanding, other orders may be used. Blocks may be performed in parallel in combinatorial logic in the criticality control circuit 34. Blocks, combinations of blocks, and / or the flow pattern as a whole may be routed over multiple clock cycles. The criticality control circuit 34 may be configured to implement the operation shown in FIG. 6.If the cache line being filled has a high critical state (e.g., a critical state that does not correspond to the lowest of the critical states, in one embodiment) (decision block 100, "yes" branch), the cache line may be inserted into the replacement data at the MRU location (block 102). If the cache line has a critical state (e.g., the lowest critical state) (decision block 100, "no" branch and decision block 104, "yes" branch), the criticality control circuit 34 may be configured to insert the cache line at a highest possible position in the replacement data (closest to the MRU), but below the positions of high critical state cache lines. Thus, if one or more high critical state cache lines are present in the replacement data (decision block 106, "yes" branch), the criticality control circuit 34 may insert the cache line at the highest position that is lower than the high critical state cache lines (block 108). Otherwise, the cache line may be inserted at the MRU location (decision block 106, "no" branch, and block 102).If the cache line being filled is not critical (decision blocks 100 and 104, "no" branches) and the fill is due to a prefetch (instruction or data) (decision block 110, "yes" branch), the prefetch may be inserted at N near the LRU (block 112), similar to discussed above with respect to FIG. 3. In one embodiment, instruction prefetches may be placed at a lower LRU position than data prefetches, but both may be placed near the LRU position. Alternatively, instruction prefetches may be placed at a higher LRU position than data prefetches, but both may be placed near the LRU, or the same LRU position may be used for both prefetch types. If the non-critical cache line is not a prefetch but an NT request (decision block 114, "yes" branch), the cache line may be inserted at position M, which in this embodiment is greater than N but near the LRU (block 116). If the non-critical cache line is a demand request (decision block 114, "no" branch) and there are critical cache lines (decision block 106, "yes" branch), the non-critical cache line may be inserted below the critical cache lines (block 108). If there are no critical cache lines in the set (decision block 106, "no" branch), the non-critical cache line may be inserted at the MRU location (block 102).The circuitry represented by decision block 106 and blocks 102 and 108 may provide a dynamic insertion point for particular cache lines, thereby preventing a "priority inversion" in the replacement data if critical cache lines could be moved down toward the LRU position by less critical cache lines in the replacement data.FIG. 7 is a flow diagram illustrating the operation of an embodiment of the criticality control circuit 34 to update the replacement data for a set based on a hit to a cache line in the criticality control circuit 34 (a stepping up of a cache line). Although the blocks are shown in a particular order for ease of understanding, other orders may be used. Blocks may be performed in parallel in combinatorial logic in the criticality control circuit 34. Blocks, combinations of blocks, and / or the flow pattern as a whole may be routed over multiple clock cycles. The criticality control circuit 34 may be configured to implement the operation shown in FIG. 7.If the hit cache line has any critical state level (decision block 120, "yes" branch), the criticality control circuit 34 may update the cache line at the MRU position (block 122). If the hit cache line is not critical (decision block 120, "no" branch) and the hit cache line is an uncommitted prefetch request (decision block 124, "yes" branch), the criticality control circuit 34 may leave the replacement data position unchanged, but may reset the prefetch bit (block 126). If the hit request is a demand or data prefetch (decision block 128, "yes" branch), the criticality control circuit 34 may maintain the priority of the critical cache lines by stepping up the hit cache line to the highest replacement data position that is below the critical cache lines (decision block 130, "yes" branch, and block 132). If there are no critical cache lines in the set, the hit cache line may be made the MRU (decision block 130, "no" branch and block 122). If the hit request is an NT request (decision block 134, "yes" branch), the hit cache line may be updated to position P, which is close to the LRU, if the hit cache line is an unsmooth prefetch, i.e., the position is unchanged (block 136). If the hit request is not an NT request (and neither of the other types of requests mentioned above), the request may be an instruction prefetch and the hit cache line may be updated to MRU (block 138).Similar to that discussed above with respect to FIG. 6, the circuitry represented by decision block 130 may provide a spare dynamic data update to prevent priority inversion between non-critical cache lines and critical cache lines. The embodiment of FIG. 6 may allow different levels of critical cache lines in the replacement data to be reordered, but may maintain the non-critical cache lines below the critical cache lines in the replacement data.Referring to FIG. 8, a flow diagram is shown illustrating the operation of one embodiment of the criticality control circuit 34 for selecting a victim cache line to be emitted when a cache miss is detected. Although the blocks are shown in a particular order for ease of understanding, other orders may be used. Blocks may be performed in parallel in combinatorial logic in the criticality control circuit 34. Blocks, combinations of blocks, and / or the flow pattern as a whole may be routed over multiple clock cycles. The criticality control circuit 34 may be configured to implement the operation shown in FIG. 8.If at least one invalid entry is present in the set (decision block 140, "yes" branch), the criticality control circuit 34 may mask all valid entries and select the LRU-next non-masked (invalid) entry (block 142). If all entries are valid (decision block 140, "no" branch), the criticality control circuit 34 may determine a corrupted pseudo random selection, similar to discussed above with respect to FIG. 4 (block 144). Based on the pseudo random selection, the criticality control circuit 34 can selectively mask all critical cache lines (block 146). If at least one non-masked valid entry is found (decision block 148, "yes" branch), the criticality control circuit 34 may select the LRU-next non-masked entry (block 142). If no entry is found (decision block 148, "no" branch), the criticality control circuit 34 may demask the lowest level of critical cache lines while still masking the higher critical cache lines (block 150). If at least one non-masked valid entry is found (decision block 152, "yes" branch), the criticality control circuit 34 may select the LRU-next non-masked entry (block 142). If no entry is found (decision block 152, "no" branch), the criticality control circuit 34 may demask all critical cache lines (block 154) and select the LRU next non-masked entry (block 142).FIG. 9 is a flow diagram illustrating the operation of another embodiment of the criticality control circuit 34 for selecting a victim cache line to be emitted when a cache miss is detected. Although the blocks are shown in a particular order for ease of understanding, other orders may be used. Blocks may be performed in parallel in combinatorial logic in the criticality control circuit 34. Blocks, combinations of blocks, and / or the flow pattern as a whole may be routed over multiple clock cycles. The criticality control circuit 34 may be configured to implement the operation shown in FIG. 9.The embodiment of FIG. 9 may use multiple corrupted pseudo-random selection methods based on different probabilities to selectively mask or not mask different subsets of the critical state until a victim is selected. Similar to the embodiment of FIG. 8, if there is at least one invalid entry in the set (decision block 160, "yes" branch), the criticality control circuit 34 may mask all valid entries and select the LRU-next non-masked (invalid) entry (block 162). If all entries are valid (decision block 160, "no" branch), the criticality control circuit 34 may determine a first corrupted pseudo-random selection based on a first probability, similar to discussed above with respect to FIG. 4 (block 164). If the selection is "yes" (decision block 168, "yes" branch), the criticality control circuit 34 may mask all critical cache lines (block 168) and determine whether at least one valid, non-masked entry has been found (decision block 170). If so (decision block 170, "yes" branch), the criticality control circuit 34 may select the LRU-next non-masked entry (block 162). If not (decision block 170, "no" branch), or if the selection was "no" (decision block 166, "no" branch), the criticality control circuit 34 may determine a second corrupted pseudo-random selection based on a second probability (block 172). If the selection is "yes" (decision block 174, "yes" branch), the criticality control circuit 34 may mask critical cache lines except for the lowest critical state (block 176) and determine whether at least one valid, non-masked entry has been found (decision block 178). If so (decision block 178, "yes" branch), the criticality control circuit 34 may select the LRU-next non-masked entry (block 162). If not (decision block 178, "no" branch), or if the selection was "no" (decision block 174, "no" branch), the criticality control circuit 34 may proceed with similar iterations, thereby masking fewer of the highest levels of the critical state until an entry is found (block 180) or until all critical lines are not masked. Once an entry is found, the criticality control circuit may select the LRU-next non-masked entry (block 162).Embodiments implementing the replacement dynamic data updates to preferentially maintain critical cache lines that are closer to the MRU than other cache lines may successfully maintain the cache lines in the LLC 14. However, once the critical cache lines are no longer useful, the same characteristics may increase the difficulty in replacing the critical cache lines with last-called cache lines that are not critical.As mentioned above, during the selection of a victim cache line for replacement, the LLC 14 may be configured to preferentially maintain cache lines identified as critical by the corresponding criticality values over cache lines identified as not critical. The LLC 14 may be configured to select the victim cache line according to replacement data that is kept from the cache separate from the criticality values (and also taking the criticality values into account). However, if the criticality control circuit 34 detects one or more indications that at least some of the cache lines identified as critical are no longer critical, the criticality control circuit 34 may be configured to end the preferably retention of the cache lines based on the one or more indications. In other words, the criticality control circuit 34 may speed up the emission of the cache lines identified as critical based on the one or more indicia (as compared to the retention that would be applied prior to the detection of the one or more indicia). For example, in one embodiment, the criticality control circuit 34 may be configured to ignore the criticality values when the victim cache line is selected to end or speed up the preferably maintenance of the critical cache lines.FIG. 10 is a flow chart illustrating the operation of one embodiment of the criticality control circuit 34 to speed up the emission of critical cache lines that are no longer being used. Although the blocks are shown in a particular order for ease of understanding, other orders may be used. Blocks may be performed in parallel in combinatorial logic in the criticality control circuit 34. Blocks, combinations of blocks, and / or the flow pattern as a whole may be routed over multiple clock cycles. The criticality control circuit 34 may be configured to implement the operation shown in FIG. 10.If critical cache lines are no longer accessed (e.g., the accessing thread(s) has / have completed execution), the critical cache lines in the replacement data may eventually move to the LRU position. Accordingly, the criticality control circuit 34 may monitor the hit rates for critical cache lines located in the N-nearest LRU locations (block 190). N can be selected in any desired manner. For example, N may be about one-fourth the number of paths in a set. Additionally, as snoop copy backs of cache lines from the LLC 14 increase (i.e., snoops cause the cache lines to be forwarded to another processor 10A- 10N), the thread accessing the critical cache lines may be migrated to another cluster of processors 10A- 10N coupled to another LLC 14 in the system (not shown in FIG. 1 ). Accordingly, the criticality control circuit 34 may monitor the snoop rates that cause cache lines to be forwarded to other agents in the system (not back to the memory controller 16) (block 192). The criticality control circuit 34 may monitor snoop forwarding from only critical cache blocks or from all cache blocks, in various embodiments. Another factor that can be monitored is derived coprocessor requests (requests from coprocessor 12) (block 194).If the cache hit rates detected via the monitoring represented by block 190 are less than a threshold (decision block 196, "yes" branch), the criticality control circuit 34 may ignore the criticality values in victim selection and LRU insertion and stepping up (block 198). Thus, the cache lines may be treated the same regardless of critical / non-critical status. Similarly, if the snoop forwarding rates exceed a threshold (decision block 200, "yes" branch), the criticality control circuit 34 may ignore the criticality values in victim selection and LRU insertion and stepping up (block 198). As inferred coprocessor requests increase (decision block 202, "yes" branch), criticality control circuit 34 may ignore the criticality values in victim selection and LRU insertion and upgrading (block 198).Another factor that may be used is when the capacitance in the MCache 40 available to the processors 10A-10N / LLC 14 falls below a threshold (e.g., as indicated by the indication CPU capacitance from the monitoring circuit 42) (decision block 204, "yes" branch, the criticality control circuit 34 treats all critical state levels as the lowest critical state (block 206). If nothing of the foregoing is true (decision blocks 196, 200, 202, and 204, "no" branches), the criticality control circuit 34 may maintain the use of criticality values in victim selection and LRU insertion and stepping up (block 208).Thus, in this embodiment, the one or more indications may include a cache hit rate below a threshold for cache lines at a plurality of least recently used locations in the replacement data and having criticality values indicative of a critical status. The one or more indications may include a rate at which snoop hits occur in the cache and cause a corresponding cache line to be forwarded in response to the snoop hit being above a threshold. In systems comprising a coprocessor coupled to the cache and configured to execute coprocessor instructions issued to the coprocessor by the one or more processors, the one or more indications may comprise memory requests issued to the cache by the coprocessor. The criticality control circuit 34 may be configured to derive the coprocessor memory requests based on prefetch requests generated by the one or more processors, which are indicated as coprocessor prefetch requests. As mentioned above, the MCache 40 may provide an indication of the capacity in the second cache attributable to the data from the LLC 14, and the control circuitry is configured to override the plurality of criticality levels with a lowest of the plurality of criticality levels based on the indication that the capacity is less than a threshold.In one embodiment, a method may include: assigning criticality values to cache lines in a cache, wherein a given criticality value corresponds to a given cache line; during selection of a victim cache line for replacement, preferably maintaining cache lines identified as critical from the corresponding criticality values over cache lines that are not identified as critical, the selection further based on replacement data that is maintained separate from the cache from the criticality values; detecting one or more indications that at least some of the cache lines identified as critical are no longer critical; and ignoring the criticality values for victim selection and replacement data update based on the one or more indications. For example, in one embodiment, the method further comprises monitoring a cache hit rate for cache lines located at a plurality of least recently used locations in the replacement data and having criticality values indicative of a critical status, and one of the one or more indications is based on the cache hit rate being below a threshold. In one embodiment, the method further comprises monitoring a rate at which snoop hits occur in the cache and cause a corresponding cache line to be forwarded in response to the snoop hit, and one of the one or more indications is based on the snoop hit rate being above a threshold. In one embodiment, the one or more indications comprise memory requests issued to the cache by a coprocessor, the coprocessor coupled to the cache and configured to execute coprocessor instructions issued to the coprocessor by one or more processors. The method may further include deriving the coprocessor memory requests based on prefetch requests generated by one or more processors indicated as coprocessor prefetch requests. The method may further comprise, in one embodiment, providing an indication from a second cache of capacity in the second cache that is attributable to data from the cache, wherein the criticality values are not critical and indicate a plurality of criticality levels; and overwriting the plurality of criticality levels with a lowest of the plurality of criticality levels based on the indication of capacity being lower than a threshold.FIG. 11 is a flow diagram illustrating the operation of one embodiment of the criticality control circuit 34 and the MCache 40 for inserting emitted cache lines from the LLC 14 into the replacement data of the MCache 40. Although the blocks are shown in a particular order for ease of understanding, other orders may be used. Blocks may be performed in parallel in combinatorial logic in the criticality control circuit 34 and / or in the MCache 40. Blocks, combinations of blocks, and / or the flow pattern as a whole may be routed over multiple clock cycles. The criticality control circuit 34 / MCache 40 may be configured to implement the operation shown in FIG. 11.If the issued cache line is a critical cache line (decision block 210, "yes" branch), the criticality control circuit 34 may generate the insert indication to the MCache 40 to insert the cache line at the MRU position (block 212). Alternatively, the MCache 40 may detect the critical status of the cache line and insert the cache line at the MRU location. If the cache line is not critical (decision block 210, "no" branch), the criticality control circuit 34 may generate and index the MCache insert LUT 36 based on the telemetry data to the CPU capacity (block 214). For example, the telemetry data to CPU capacity may indicate the average number of paths of MCache 40 available for cache lines from processors 10A-10N / LLC 14. The index may be generated based on the average number of paths being in different ranges. For example, up to one eighth of the number of paths, one eighth to one quarter of the number of paths, one quarter to one half of the number of paths, and more than one half of the number of paths may be the index for a two-bit insert indication. The criticality control circuit 34 may generate the insertion hint from the indexed entry in the MCache insertion LUT 36 (block 216).When the insertion control circuitry and LUT 38 receives the emitted cache block, the insertion hint may be used as an index for the LUT 38 and the insertion position may be read from the table (block 218). The insertion control circuit 38 may modify the insertion position based on whether or not MCache paths are turned off for power saving. That is, each powered down path occupies an LRU position in the replacement data because it cannot be used. If the insertion position were to be in one of the N LRU positions, where N is the number of paths shut down, the insertion position may be increased to N (block 220). The MCache 40 may assign an entry for the cache line and update the entry with the cache line (block 222), and the cache line, optionally emitted from the MCache 40, is written to the memory 18 when modified with respect to the copy in the memory 18. The MCache 40 may update the replacement data to indicate the allocated entry at the insertion position (block 224).In one embodiment, MCache 40 may also support a dynamic insert position for non-critical cache lines from processors 10A-10N / LLC14. For example, the MCache 40 may determine the MRU-next, non-critical cache line (not including the cache line for which the insertion point is detected), referred to in this paragraph as position H. If there are no non-critical cache lines, the MCache 40 may insert the cache line at the adjusted insertion position described in the preceding paragraph. However, if valid non-critical cache lines are present in the MCache 40 and the inserted cache line is already at an MRU-closer position than position H, the MCache 40 may insert the cache line at the position closer to the MRU than position H. Otherwise, the MCache 40 may insert the cache line at position H. The MCache 40 may update the replacement data to indicate the allocated entry at the insertion position (block 224).FIG. 12 is a block diagram of an embodiment of a system including a system on a chip (SOC) 300 coupled to a memory 18. As the name dictates, the components of the SOC 300 may be integrated on a single semiconductor substrate as an integrated circuit "chip.". In the illustrated embodiment, the components of the SOC 300 include a processor cluster 304, another processor cluster 206, one or more peripheral components such as the peripheral components 308A- 308B (briefly, "peripheral devices"), the memory controller 16, and a communication fabric 312. Components 304, 306, 308A- 308B, and 16 may all be coupled to communication fabric 312. The memory controller 16 may be coupled to the memory 18 during use. In some embodiments, there may be more than one memory controller coupled to a corresponding memory. The memory address space may be allocated in any desired manner via the memory controllers in such embodiments. In the illustrated embodiment, processor cluster 304 may include a plurality of processors (P) 10A- 10N. The processors 10A- 10N may form the central processing units (CPU(s)) of the SOC 300. Processor cluster 304 may further include one or more coprocessors (e.g., coprocessor 12 in FIG. 12 ). The processor cluster 304 may further include the LLC 14. Processor cluster 306 may be similar to processor cluster 304. Thus, the SOC 300 may be an implementation of the system shown in FIG. 1.The memory controller 16 may generally include the circuitry to receive memory operations from the other components of the SOC 300 and access the memory 18 to complete the memory operations. The memory controller 12 may be configured to access any type of memory 18. For example, the memory 18 may be static RAM (SRAM) or dynamic RAM (DRAM) such as synchronous DRAM (SDRAM) including double data rate DRAM (DDR, DDR2, DDR3, DDR4, etc.). Low power / mobile versions of DDR DRAM may be supported (e.g., LPDDR, mDR, etc.). The memory controller 16 may include queues for memory operations to order (and potentially reorder) the operations and provide the operations to the memory 18. The memory controller 16 may further include data buffers to store write data awaiting writing to the memory and read data awaiting returning to the source of the memory operation. In some embodiments, the memory controller 16 may include a memory cache (MCache) 40 to store recently accessed memory data. In SOC implementations, for example, the MCache 40 may reduce power consumption in the SOC by avoiding re-access to data from the memory 16 when it is expected to be re-accessed soon. In some cases, unlike private caches, such as LLC 14 or caches in processors 10A- 10N that service only certain components, MCache 40 may also be referred to as a system cache. Additionally, in some embodiments, a system cache need not be located within memory controller 16.The peripherals 308A- 308B may be any set of additional hardware functionality included in the SOC 300. For example, the peripheral devices 308A- 308B may include video peripherals such as one or more graphics processors (GPUs), an image signal processor configured to process image capture data from a camera or other image sensor, video encoder / decoders, scalers, rotators, display controllers, mixers, etc. The peripheral devices may include audio peripheral devices such as microphones, speakers, interfaces for microphones and speakers, audio processors, digital signal processors, mixers, etc. The peripheral devices may include interface controllers for various interfaces external to the SOC 100, including interfaces such as universal serial bus (USB), peripheral component interconnect (PCI), including PCI express (PCIe), serial and parallel ports, etc. The connection to the external device in FIG. 12 is illustrated by a dashed arrow extending external to the SOC 300. The peripherals may include network peripherals such as media access controllers (MACs). Any set of hardware may be included.The communication structure 312 may be any communication link and protocol for communicating between the components of the SOC 300. The communication fabric 312 may be bus-based, including shared bus configurations, crossbar configurations, and hierarchical buses with bridges. The communication structure 312 may also be packet-based and may be hierarchical with bridges, crossbar, point-to-point, or other connections.Note that the number of components of the SOC 300 (and the number of subcomponents of those shown in FIG. 12, such as the processors 10A- 10N in each processor cluster 304 and 306, may vary from embodiment to embodiment. In addition, the number of processors 10A- 10N in one processor cluster 304 may be different than the number of processors 10A- 10N in the other processor cluster 306. There may be more or less of each component / sub-component than the number shown in FIG. 12.Based on the foregoing, in one embodiment, a system may include one or more processors configured to issue memory requests to access a memory system; and a cache coupled to the one or more processors and configured to cache data from the memory system for access by the one or more processors. The cache may include a control circuit configured to assign criticality values to the cache lines based on a plurality of factors at a time the cache lines are filled into the cache. During filling of a given cache line, the control circuitry may be configured to present a given cache line at a selected location in the replacement data for the cache based on the criticality value associated with the given cache line. The control circuitry may be configured to select a victim cache line to be dispatched from the cache based on the replacement data. The control circuitry may be configured to selectively prevent selection of cache lines having criticality values indicative of a critical status as a victim cache line based on a probability.In one embodiment, the system further comprises a second cache coupled to the cache and configured to cache data from the storage system for the cache and for one or more other cache access agents in the system. The second cache is configured to store the victim cache line and maintain an indication of the criticality value assigned to the victim cache line by the control circuitry. In one embodiment, the system further comprises a memory controller configured to control one or more memory devices that form at least a portion of system memory, and the memory controller includes the second cache. In one embodiment, the second cache may be configured to provide the criticality value with the victim cache line in a fill into the cache based on another memory request made after the victim cache line is issued from the cache. In one embodiment, the second cache is configured to maintain second replacement data; and the second cache may be configured to issue cache lines from the second cache based on the second replacement data. A starting position of the victim cache line in the second replacement data may be based on the criticality value. In one embodiment, the system includes monitoring circuitry coupled to the second cache and configured to provide a capacity indication in the second cache that is attributable to data from the cache. The cache may be configured to generate an insert hint to be transmitted with the victim cache line based on the indication of capacity. For example, the cache may include a table coupled to the control circuitry that associates portions of the capacity indication with the values for the insert hint. In one embodiment, the second cache includes a second table. The second cache may be configured to select an entry in the second table based on the insert hint. The second table may be configured to output an insertion point indication from the selected entry. In one embodiment, the criticality status may include critical and not critical. In one embodiment, the criticality status may further indicate one or more criticality levels assigned to a critical cache line. In one embodiment, the control circuitry may be configured to update the replacement data based on a request that encounters a second given cache line in the cache. The replacement data may be updated by moving a position of a second entry storing the second given cache line closer to a most recently invoked position based on the criticality value assigned to the second given cache line and the criticality values of other cache lines represented in the replacement data. For example, in the case that the criticality value assigned to the second given cache line is lower than the criticality values of one or more other cache lines shown in the replacement data, the control circuitry may be configured to update the replacement data such that the second given cache line is shown at a second location below locations occupied by the one or more other cache lines.In one embodiment, the control circuitry is configured to monitor a cache hit rate for cache lines at a plurality of low positions in the replacement data and to display critical status criticality values. The control circuitry may be configured to ignore the criticality values for victim selection and replacement data update based on the cache hit rate being below a threshold. In one embodiment, the control circuitry may be configured to monitor a snoop hit rate for snoops that cause forwarding of cache lines from the cache. The control circuitry may be configured to ignore the criticality values for victim selection and replacement data update based on the snoop hit rate exceeding a threshold.Computer SystemReferring next to FIG. 13, shown is a block diagram of an embodiment of a system 700. In the illustrated embodiment, system 700 includes at least one instance of a system on a chip (SOC) 706 coupled to one or more peripheral devices 704 and external memory 702. A power supply (PMU) 708 is provided that supplies supply voltages to the SOC 706 as well as one or more supply voltages to the memory 702 and / or the peripheral devices 704. In some embodiments, more than one instance of the SOC may be included (and more than one memory 702 may also be included). The memory 702 may include the memory 18 illustrated in FIGS. 1 and 12, in one embodiment. The SOC 706 may be an instance of the SOC 300 illustrated in FIG. 12, in one embodiment.The peripheral devices 704 may include any desired circuitry depending on the type of system 700. For example, in one embodiment, the system 704 may be a mobile device (e.g., a personal digital assistant (PDA), a smart phone, etc.), and the peripheral devices 704 may include devices for various types of wireless communication, such as WiFi, Bluetooth, cellular, global positioning system, etc. The peripheral devices 704 may also include additional storage including RAM storage, solid state storage, or disk storage. The peripheral devices 704 may include user interface devices such as a display screen, including touch or multi-touch display screens, keyboard or other input devices, microphones, speakers, etc. In other embodiments, system 700 may be any type of computing system (e.g., desktop personal computer, laptop, workstation, netp, etc.).External memory 702 may include any type of memory. For example, external memory 702 may be SRAM, dynamic RAM (DRAM) such as synchronous DRAM (SDRAM), DOUBLE DATA RATE SDRAM (DDR, DDR2, DDR3, etc.), RAMBUS DRAM, low power versions of DDR DRAM (e.g., LPDDR, mD, etc.), etc. The external memory 702 may include one or more memory modules to which the memory devices are connected, such as single inline memory modules (SIMMs), dual inline memory modules (DIMMs), etc. Alternatively, the external memory 702 may include one or more memory devices connected to the SOC 706 in a chip-on-chip or package-on-package implementation.As illustrated, the system 700 is shown to find application in a wide range of ranges. For example, system 700 may be utilized as part of the chips, circuitry, components, etc., of a desktop computer 710, laptop computer 720, tablet computer 730, cellular or cellular telephone 740, or television 750 (or a set-top box coupled to a television). Also illustrated are a smart watch and health monitoring device 760. In some embodiments, a smart watch may include a variety of functions related to general data processing. For example, a smart watch may provide access to email, a cellular phone service, a user calendar, and so forth. In various embodiments, a health monitoring device may be a dedicated medical device or otherwise include dedicated health related functionality. For example, a health monitoring device may monitor a user's vital parameters, track a user's proximity to other users for purposes of epidemiological distance preservation, contact tracking, provide communication to an emergency service in the event of a health emergency, and so forth. In various embodiments, the above-mentioned smart watch may include some or no health monitoring-related functions. Other wearable devices are also contemplated, such as devices worn around the neck, devices implantable in the human body, glasses configured to provide an augmented and / or virtual reality experience, and so forth.The system 700 may further be used as part of a cloud-based service / services 770. For example, the aforementioned devices and / or other devices may access computing resources in the cloud (i.e., remote hardware and / or software resources). Still further, the system 700 may be utilized in one or more home devices other than those mentioned above. For example, devices within the home may monitor and detect conditions that require attention. For example, various devices within the home (e.g., a refrigerator, a refrigeration system, etc.) may monitor the status of the device and provide a warning to the home owner (or, for example, a repair facility) should a particular event be detected. Alternatively, a thermostat may monitor temperature in the home and may automate settings of a heating / cooling system based on a history of reactions to various conditions by the home owner. Also illustrated in FIG. 13 is the application of the system 700 in different traffic carriers. For example, system 700 may be used in aircraft, train, bus, rental car, passenger car, watercraft, private boats to cruise ships, (rental or own) scooters control and / or entertainment systems, and so forth. In various cases, the system 700 may be used to provide automated guidance (e.g., self-driving vehicles), general system control, and others. These many other embodiments are possible and are contemplated. It should be noted that the devices and applications illustrated in FIG. 13 are illustrative only and are not intended to be limiting. Other devices are possible and are contemplated.Computer readable storage mediumReferring now to FIG. 14, a block diagram of one embodiment of a computer readable storage medium 800 is shown. Generally speaking, a computer-accessible storage medium may include any storage medium that can be accessed by a computer during use to provide instructions and / or data to the computer. For example, a computer-accessible storage medium may include storage media such as magnetic or optical media, e.g., disks (fixed or portable), tapes, CD-ROM, DVD-ROM, CD-R, CD-RW, DVD-R, DVD-RW, or Blu-ray. Storage media may further include volatile or non-volatile storage media such as RAM (e.g., synchronous dynamic RAM (SDRAM), Rambus DRAM (RDRAM), static RAM (SRAM), etc.), ROM, or flash memory. The storage media may be physically enclosed within the computer for which the storage media provides instructions / data. Alternatively, the storage media may be connected to the computer. For example, the storage media may be connected to the computer via a network or a wireless connection such as network memory. The storage media may be connected via a peripheral interface such as the Universal Serial Bus (USB). Generally, the computer-accessible storage medium 800 may store data in a non-transitory manner, wherein non-transitory in this context may refer to the instructions / data not being transmitted on a signal. For example, the non-transitory storage may be volatile (and lose the stored instructions / data in response to a shutdown), or may be nonvolatile.The computer-accessible storage medium 800 in FIG. 14 may store a database 804 representative of the SOC 300. Generally, database 804 may be a database that can be read by a program and used directly or indirectly to fabricate the hardware including SOC 300. For example, the database may be a behavioral level description or a register transfer level (RTL) description of hardware functionality in a high level design (HDL) language such as Verilog or VHDL. The description may be read by a synthesis tool that can synthesize the description to generate a netlist comprising a list of gates from a synthesis library. The netlist includes a set of gates that also represent the functionality of the hardware that includes the SOC 300. The netlist can then be placed and routed to generate a dataset that describes geometric shapes to apply to masks. The masks may then be used in various semiconductor fabrication steps to fabricate a semiconductor circuit or circuits corresponding to SOC 300. Alternatively, database 804 on computer-accessible storage medium 800 may be the netlist (with or without the synthesis library) or record as desired.While computer-accessible storage medium 800 stores a representation of SOC 300, other embodiments may carry a representation of any portion of SOC 100, including any subset of the components shown in FIG. 12, as desired. Moreover, database 804 may represent processors 10A- 10N, coprocessor 12, or both, as shown in FIG. 1, and may further represent LLC 14 and / or memory controller 16. Database 804 may represent any portion of the foregoing.Various embodiments are contemplated as set forth in the following numbered examples: 1stsystem comprising:one or more processors configured to issue memory requests for accessing a memory system; anda cache coupled to the one or more processors and configured to cache data from the memory system for access by the one or more processors, wherein:the cache includes a control circuit configured to assign criticality values to cache lines, wherein a given criticality value corresponds to a given cache line;the cache is configured to, during selection of a victim cache line for replacement, preferably maintain cache lines identified as critical from the corresponding criticality values over cache lines not identified as critical, the cache further configured to select the victim cache line according to replacement data maintained separate from the cache from the criticality values;the control circuitry is configured to recognize one or more indications that at least some of the cache lines identified as critical are no longer critical; andthe control circuitry is configured to terminate the preferred retention of the cache lines based on the one or more indications.2. The system of example 1, wherein the control circuitry is configured to monitor a cache hit rate for cache lines located at a plurality of least recently used locations in the replacement data and having criticality values indicative of a critical status, and wherein one of the one or more indications is based on the cache hit rate being below a threshold. 3. the system of example 1 or 2, wherein the control circuitry is configured to monitor a rate at which snoop hits occur in the cache and cause a corresponding cache line to be forwarded in response to the snoop hit, and wherein one of the one or more indications is based on the snoop hit rate being above a threshold. 4. the system of any of Examples 1 to 3, further comprising a coprocessor coupled to the cache and configured to execute coprocessor instructions issued to the coprocessor by the one or more processors, and wherein the one or more indications comprise memory requests issued to the cache by the coprocessor. 5. the system of example 4, wherein the control circuitry is configured to derive the coprocessor memory requests based on prefetch requests generated by the one or more processors, indicated as coprocessor prefetch requests. 6.The system of any preceding example, further comprising a second cache coupled to the cache, wherein the second cache is configured to provide an indication of capacity in the second cache attributable to the data from the cache, and wherein the criticality values are not critical and indicate a plurality of criticality levels, and wherein the control circuitry is configured to override the plurality of criticality levels with a lowest of the plurality of criticality levels based on the indication that the capacity is less than a threshold. 7. the system of any preceding example, wherein the control circuit is configured to terminate the preferred retention by at least partially ignoring the criticality values for the victim selection and the replacement data update. 8. a system comprising:one or more processors configured to issue memory requests for accessing a memory system; anda cache coupled to the one or more processors and configured to cache data from a memory system for access by the one or more processors, the cache comprising control circuitry, and wherein:the control circuitry is configured to assign criticality values to the cache lines, wherein a given criticality value corresponds to a given cache line;the cache is configured to, during selection of a victim cache line as a replacement, preferably cache lines identified as critical from the corresponding criticality values,retain cache lines that are not identified as critical, wherein the cache is further configured to select the victim cache line according to replacement data kept separate from the cache from the criticality values;the control circuitry is configured to recognize one or more indications that at least some of the cache lines identified as critical are no longer critical; andthe control circuitry is configured to speed up the issuance of the cache lines identified as critical based on the one or more indications.9. The system of example 8, wherein the control circuitry is configured to monitor a cache hit rate for cache lines located at a plurality of least recently used locations in the replacement data and having criticality values indicative of a critical status, and wherein one of the one or more indications is based on the cache hit rate being below a threshold. 10. the system of example 8 or 9, wherein the control circuitry is configured to monitor a rate at which snoop hits occur in the cache and cause a corresponding cache line to be forwarded in response to the snoop hit, and wherein one of the one or more indications is based on the snoop hit rate being above a threshold. 11 The system of any of Examples 8-10, wherein the one or more indications comprise memory requests issued to the cache by a coprocessor, the coprocessor coupled to the cache and configured to execute coprocessor instructions issued to the coprocessor by the one or more processors. 12th The system of Example 11, wherein the control circuitry is configured to derive the coprocessor memory requests based on prefetch requests generated by the one or more processors, indicated as coprocessor prefetch requests. 13.The system of any of Examples 8-12, further comprising a second cache configured to provide an indication of capacity in the second cache that is attributable to data from the cache, and wherein the criticality values are not critical and indicate a plurality of criticality levels, and wherein the control circuitry is configured to override the plurality of criticality levels with a lowest of the plurality of criticality levels based on the indication of capacity being less than a threshold. 14. system according to any of Examples 8-13, wherein the control circuitry is configured to speed up the emission by at least partially ignoring the criticality values for the victim selection and the replacement data update. 15. A method comprising:assigning criticality values to cache lines in a cache, wherein a given criticality value corresponds to a given cache line;while selecting a victim cache line for replacement, preferably maintaining cache lines identified as critical from corresponding criticality values versus cache lines not identified as critical, the selection further based on replacement data maintained separate from the cache from the criticality values;detecting one or more indicia that at least some of the cache lines identified as being critical are no longer critical; and ignoring the criticality values for victim selection and replacement data update based on the one or more indicia.16. The method of example 15, further comprising monitoring a cache hit rate for cache lines located at a plurality of longest unused locations in the replacement data and having criticality values indicative of a critical status, and wherein one of the one or more indications is based on the cache hit rate being below a threshold. 17. the method of example 15 or 16, further comprising monitoring a rate at which snoop hits occur in the cache and cause a corresponding cache line to be forwarded in response to the snoop hit, and wherein one of the one or more indications is based on the snoop hit rate being above a threshold. 18. The method of any of Examples 15-17, wherein the one or more indications comprise memory requests issued to the cache by a coprocessor, the coprocessor coupled to the cache and configured to execute coprocessor instructions issued to the coprocessor by one or more processors. 19.The method of Example 18, further comprising deriving the coprocessor memory requests based on prefetch requests generated by one or more processors, indicated as coprocessor prefetch requests. 20.The method of any one of Examples 15 to 19, further comprising:providing an indication from a second cache of capacity in the second cache attributable to data from the cache, wherein the criticality values are not critical and indicate a plurality of criticality levels; andoverwriting the plurality of criticality levels with a lowest one of the plurality of criticality levels based on the indication of the capacity being lower than a threshold.

Claims

A system comprising: one or more processors (10A-10N) configured to issue memory requests for accessing a memory system (18); and a cache (32) coupled to the one or more processors (10A-10N) and configured to cache data from the memory system (18) for access by the one or more processors (10A-10N), wherein: the cache (32) comprises a control circuit (34) configured to assign criticality values to cache lines, wherein a given criticality value corresponds to a given cache line; the cache (32) is configured to, during selection of a victim cache line as a replacement, preferably maintain cache lines identified as critical from the corresponding criticality values over cache lines not identified as critical, the cache (32) further configured to select the victim cache line according to replacement data maintained separate from the cache (32) from the criticality values; the cache (32) is configured to monitor a cache hit rate for cache lines having criticality values indicating a critical status; the control circuitry (34) is configured to recognize one or more indications that at least some cache lines identified as critical are no longer critical, wherein one of the one or more indications is based on the cache hit rate being below a threshold; and the control circuitry (34) is configured to end the preferred retention of the cache lines based on the one or more indications.The system of claim 1, wherein to monitor the cache hit rate, the control circuitry (34) is configured to monitor the hit rate for cache lines at a plurality of at least recently used positions in the replacement data and having criticality values indicative of a critical status.The system of claim 1, wherein the control circuitry (34) is configured to monitor a rate at which snoop hits occur in the cache (32) and cause a corresponding cache line to be forwarded in response to a given snoop hit, and wherein one of the one or more indications is based on the snoop hit rate being above a threshold.The system of claim 1, further comprising a coprocessor (12) coupled to the cache (32) and configured to execute coprocessor instructions issued by the one or more processors (10A-10N) to the coprocessor (12), and wherein the one or more indicia comprises memory requests issued by the coprocessor (12) to the cache (32).The system of claim 4, wherein the control circuitry (34) is configured to derive the coprocessor memory requests based on prefetch requests generated by the one or more processors (10A-10N), indicated as coprocessor prefetch requests.The system of claim 1, further comprising a second cache (40) coupled to the cache (32), wherein the second cache (40) is configured to provide an indication of capacity in the second cache (40) that data from the cache (32) can be associated, and wherein the criticality values are not critical and indicate a plurality of criticality levels, and wherein the control circuitry (34) is configured to override the plurality of criticality levels with a lowest of the plurality of criticality levels based on the indication that the capacity is less than a threshold.The system of claim 1, wherein the control circuit (34) is configured to terminate the preferred retention by at least partially ignoring the criticality values for the victim selection and the replacement data update.A method comprising: assigning criticality values to cache lines in a cache (32), a given criticality value corresponding to a given cache line; during selection of a victim cache line for replacement, preferably maintaining cache lines identified as critical from the corresponding criticality values versus cache lines not identified as critical, the selection further based on replacement data maintained separate from the cache (32) from the criticality values; monitoring a cache hit rate for cache lines with criticality values indicating a critical status; Detecting one or more indications that at least some cache lines identified as critical are no longer critical, wherein one of the one or more indications is based on the cache hit rate being below a threshold; and ignoring the criticality values for the victim selection and the replacement data update based on the one or more indications.The method of claim 8, wherein monitoring the cache hit rate for cache lines is at a plurality of at least recently used positions in the replacement data and with criticality values indicative of a critical status.The method of claim 8, further comprising monitoring a rate at which snoop hits occur in the cache (32) and cause a corresponding cache line to be forwarded in response to a given snoop hit, and wherein one of the one or more indications is based on the snoop hit rate being above a threshold.The method of claim 8, wherein the one or more indicia comprises memory requests issued to the cache (32) by a coprocessor (12), the coprocessor (12) being coupled to the cache (32) and configured to execute coprocessor instructions issued to the coprocessor (12) by one or more processors (10A-10N).The method of claim 11, further comprising deriving the coprocessor memory requests based on prefetch requests generated by one or more processors (10A-10N) that are indicated as coprocessor prefetch requests.The method of claim 8, further comprising: providing an indication from a second cache (40) of capacity in the second cache (40) attributable to data from the cache (32), wherein the criticality values are not critical and indicate a plurality of criticality levels; and overwriting the plurality of criticality levels with a lowest of the plurality of criticality levels based on the indication of capacity being lower than a threshold.

Images