Cache Associativity Allocation
By categorizing cache requests and allocating associativity using a binary tree traversal with locked directions, the technique optimizes cache efficiency and prevents workload dominance, enhancing performance in multi-threading scenarios.
Patent Information
- Application Number
- JP2024526934
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-12-21
- Filing Date
- 2022-12-14
- Publication Date
- 2024-12-05
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing cache systems struggle to efficiently allocate associativity to different categories of cache requests, leading to dominance by certain workloads and starvation of others, particularly in multi-threading scenarios.
The technique differentiates cache requests into categories by assigning a portion of associativity to each category, using a binary tree traversal algorithm with locked traversal directions to restrict cache line allocation based on workload or thread type, ensuring fair access and improved forward progression.
This approach enhances cache efficiency by allowing specific categories to dominate cache lines, preventing starvation and improving performance in multi-threading environments by optimizing cache usage based on request categories.
Smart Images

Figure 2024544866000001_ABST
Abstract
Description
[Technical field]
[0001] (Related Applications) This application claims priority to U.S. patent application Ser. No. 17 / 557,731, entitled "Cache Associativity Allocation," filed Dec. 21, 2021, the entire disclosure of which is incorporated herein by reference. [Background technology]
[0002] A cache is a hardware or software component that stores data (at least temporarily) so that future requests for the data can be served faster than if the data were served from main memory. A "cache hit" occurs when the requested data can be found in the cache, while a "cache miss" occurs when the requested data cannot be found in the cache. A cache miss occurs, for example, in scenarios where the requested data has not yet been loaded into the cache, or when the requested data has been evicted from the cache prior to the request. A cache replacement policy defines rules for selecting which cache line of a cache to evict so that the requested data can be loaded into the selected cache line in response to a cache miss.
[0003] The detailed description will now be made with reference to the accompanying drawings. [Brief description of the drawings]
[0004] [Figure 1] FIG. 2 is a block diagram of a non-limiting exemplary system having a controller with a cache and an associativity allocator in accordance with some embodiments. [Diagram 2] 1 illustrates a non-limiting example of an associativity locator allocating portions of a cache's associativity to categories of cache requests. [Diagram 3]FIG. 2 illustrates a non-limiting example of a cache replacement policy tree structure being traversed according to a cache replacement policy traversal algorithm to select a cache line. [Figure 4] 1 illustrates a non-limiting example of an embodiment in which a cache replacement policy is modified to allocate a portion of the cache's associativity to categories of cache requests. [Diagram 5] FIG. 2 illustrates a procedure in an exemplary embodiment for allocating portions of a cache's associativity to categories of cache requests. [Figure 6] FIG. 2 illustrates a procedure in an exemplary embodiment for splitting cache associativity and allocating portions of the cache associativity to different categories of cache requests. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0005] (overview) The associativity of a cache defines the set of cachelines of the cache into which data is permitted to be loaded in response to a cache miss. A cache that is "fully associative," for example, allows data to be loaded into any cacheline of the cache. However, a fully associative cache may be dominated by a particular workload with a large number of requests to the cache, making it difficult for other workloads to utilize the cache.
[0006] To solve this problem, cache associativity allocation is described herein. The described technique allocates portions of the associativity of a cache to different categories of cache requests, such as by allocating a first portion of the associativity to a first category of cache requests, a second portion of the associativity to a second category of cache requests, and so on. The different categories correspond, for example, to different workloads or threads executed by the cache client. For example, the first category is associated with requests corresponding to the first workload, and the second category is associated with requests corresponding to the second workload. A portion of the associativity of a cache is allocated to a particular category of cache requests by reserving a subset of cachelines of the cache for the particular category, such that data associated with the cache requests of the category is loaded into the reserved subset of cachelines, for example, in response to a cache miss.
[0007] In one or more embodiments, a cache replacement policy that controls loading of data into a cache in response to a cache miss includes a binary tree having leaf nodes corresponding to cache lines of the cache, and also includes a pseudo-least recently used algorithm that is utilized to traverse the binary tree to select cache lines to evict in response to a cache miss. To allocate a portion of the cache's associativity to the categories, in this embodiment, the pseudo-least recently used algorithm is modified by "locking" or otherwise setting traversal direction indicators of nodes of the binary tree in a first direction (e.g., left) for a first category of requests and in a second direction (e.g., right) for a second category of requests. In this manner, a first category of cache requests is limited to loading data into cache lines that correspond to leaf nodes oriented to the left of the locked node of the binary tree, and a second category of cache requests is limited to loading data into cache lines that correspond to leaf nodes oriented to the right of the locked node of the binary tree.
[0008] By allocating portions of the associativity of a cache to different categories of cache requests, the described techniques restrict which cache lines data is permitted to be loaded into based on the category of request associated with the data, e.g., the workload to which the request corresponds. By further restricting the cache lines into which data is permitted to be loaded based on the category, the described techniques prevent a particular category of cache requests from dominating usage of all cache lines of a cache that would otherwise be permitted by a traditional cache replacement policy.
[0009] In some aspects, techniques described herein relate to a method that includes assigning a portion of an associativity of a cache to a category of a cache request, the portion of the associativity corresponding to a subset of cachelines of the cache; receiving a request to access the cache; assigning a cacheline of the subset of cachelines to the request based on a category associated with the request; and loading data corresponding to the request into a cacheline of the subset of cachelines.
[0010] In some aspects, the techniques described herein relate to a method where allocating a portion of the associativity includes locking traversal of at least one node of a tree structure in a direction, where the tree structure is traversed to identify which cache line to allocate to the request, and where traversal is locked in the direction for requests associated with the category.
[0011] In some aspects, the techniques described herein relate to a method further including allocating additional portions of the cache's associativity to additional categories of cache requests, the additional portions of the associativity being allocated by locking traversal of at least one node of the tree structure in a different direction for requests associated with the additional categories.
[0012] In some aspects, the technology described herein relates to a method in which the tree structure is a binary tree that is traversed according to a pseudo-least recently used algorithm to identify which cache line to allocate to the request.
[0013] In some aspects, the techniques described herein relate to a method that further includes allocating cache lines of the subset of cache lines and loading data corresponding to the request by traversing the tree structure.
[0014] In some aspects, the techniques described herein relate to a method further including allocating an additional portion of the associativity of the cache to an additional category of cache requests, the additional portion of the associativity corresponding to an additional subset of cachelines of the cache; receiving an additional request associated with the additional category for accessing the cache; allocating a cacheline of the additional subset of cachelines to the additional request; and loading additional data corresponding to the additional request into the cacheline of the additional subset of cachelines.
[0015] In some aspects, the techniques described herein relate to a method in which a category corresponds to a first workload or thread and an additional category corresponds to a second workload or thread.
[0016] In some aspects, the techniques described herein relate to a method that further includes determining that a category associated with the request corresponds to a category of the cache request.
[0017] In some aspects, the techniques described herein relate to a method in which a category of a cache request is associated with at least one of a workload or thread of the cache request, an origin of the cache request, a destination of the cache request, or a characteristic of the cache request.
[0018] In some aspects, the techniques described herein relate to a method in which allocating a portion of a cache's associativity to a category of cache requests occurs in response to a triggering event, the triggering event including any of launching an application, initializing a workload or thread, or determining that cache usage exceeds a threshold usage.
[0019] In some aspects, the technology described herein relates to a method in which data corresponding to a request is obtained from a data store.
[0020] In some aspects, the technology described herein relates to a method, wherein the data store comprises a virtual memory.
[0021] In some aspects, the techniques described herein relate to a system that includes a cache partitioned into cachelines; and a controller for assigning a portion of the cache's associativity to a category of cache request, the portion of the associativity corresponding to a subset of the cachelines, assigning a cacheline of the subset of cachelines to the request based on the category of the request, and loading data corresponding to the request into a cacheline of the subset of cachelines.
[0022] In some aspects, the techniques described herein relate to a system in which a controller allocates a portion of a cache's associativity by locking traversal of at least one node of a tree structure in a certain direction, the tree structure being traversed to identify which cache line to allocate to a request, and traversal being locked in the direction for requests associated with a category.
[0023] In some aspects, the techniques described herein relate to a system in which the controller is further configured to allocate additional portions of the cache's associativity to additional categories of cache requests, the additional portions of the associativity being allocated by locking traversal of at least one node of the tree structure in a different direction for requests associated with the additional categories.
[0024] In some aspects, the technology described herein relates to a system in which the tree structure is a binary tree that is traversed according to a pseudo-least recently used algorithm to identify which cache line to allocate to the request.
[0025] In some aspects, the technology described herein relates to a system in which a cache is connected to an external memory.
[0026] In some aspects, the technology described herein relates to a system that includes a server, a personal computer, or a mobile device.
[0027] In some aspects, techniques described herein relate to a method that includes dividing a cache associativity into at least a first portion and a second portion; allocating the first portion of the cache associativity to a first category of cache requests, whereby the first category of cache requests are restricted to loading data using the first portion of the cache associativity in response to a cache miss; and allocating the second portion of the cache associativity to a second category of cache requests, whereby the second category of cache requests are restricted to loading data using the second portion of the cache associativity in response to a cache miss.
[0028] In some aspects, techniques described herein relate to a method where allocating a first portion of the cache's associativity to a first category of cache requests allows the first category of cache requests to access both the first portion of the cache's associativity and the second portion of the associativity, but prevents the first category of cache requests from loading data using the second portion of the cache's associativity in response to a cache miss, and where allocating the second portion of the cache's associativity to a second category of cache requests allows the second category of cache requests to access both the first portion of the cache's associativity and the second portion of the associativity, but prevents the second category of cache requests from loading data using the first portion of the cache's associativity in response to a cache miss.
[0029] 1 is a block diagram of a non-limiting exemplary system 100 having a cache and a controller with an associativity locator, according to some embodiments. In particular, the system includes a cache 102, a cache client 104, a data store 106, and a controller 108, which includes an associativity locator 110 and a cache replacement policy 112. In accordance with the described techniques, the cache 102, the cache client 104, and the data store 106 are coupled to one another via wired or wireless connections. Exemplary wired connections include, but are not limited to, a bus connecting two or more of the cache 102, the cache client 104, and the data store 106. Examples of the system 100 include, by way of example and not limitation, personal computers, laptops, desktops, servers, game consoles, set-top boxes, tablets, smartphones, mobile devices, and other computing devices.
[0030] A cache 102 is a hardware or software component that stores data (e.g., at least temporarily) so that future requests for the data are served faster from the cache 102 than from the data store 106. In one or more embodiments, the cache 102 is at least one of smaller than the data store 106, faster at serving data to the cache clients 104 than the data store 106, and / or more efficient at serving data to the cache clients 104 than the data store 106. Additionally or alternatively, the cache 102 is located closer to the cache clients 104 than the data store 106. It should be appreciated that in various embodiments, the cache 102 has additional or different characteristics that make serving at least some data to the cache clients 104 from the cache 102 more advantageous than serving such data from the data store 106.
[0031] In one or more embodiments, cache 102 is a memory cache, such as a particular level of cache (e.g., an L1 cache) that is included in a hierarchy of multiple cache levels (e.g., L0, L1, L2, L3, L4). In some variations, cache 102 is a hardware component incorporated into and used by cache client 104. In other examples, cache 102 is implemented at least partially in software, such as in at least one scenario in which cache client 104 is a web browser or web server. Also, cache 102 can be implemented in different manners without departing from the spirit or scope of the described technology.
[0032] A cache client 104 is a component that requests access to data in order to perform one or more operations related to such data. Examples of cache clients 104 include, but are not limited to, central processing units, parallel accelerated processors (e.g., graphics processing units), digital signal processors, hardware accelerators, operating systems, web browsers, web servers, applications, and lower-level caches (e.g., lower levels of the cache hierarchy than cache 102), to name a few.
[0033] In various embodiments, the cache client 104 provides a request 114 for access to data. By way of example, the request 114 is a request for write access to the data or a request for read access to the data. According to the described techniques, the request 114 is received to access the cache to attempt to find the data in the cache 102. For example, the request 114 is received by the controller 108. In response to the request 114, for example, the controller 108 searches the cache 102 to determine whether the data is stored in the cache 102. If the controller 108 identifies, by searching the cache 102, that the data is stored in the cache 102, the controller 108 then provides access to the data in the cache 102. As described herein, a "cache hit" occurs when the controller 108 is able to identify that the data identified by the request 114 is stored in the cache 102. If the request 114 is for write access, upon a cache hit, the controller 108 modifies (e.g., updates) the data in the cache 102 identified by the request 114. If the request 114 is for a read access, then upon a cache hit, the controller 108 retrieves the data in the cache 102 identified by the request 114. In the illustrated example, the data retrieved from the cache 102 based on the request 114 is shown as cache data 116. The controller 108 provides the cached data 116 to the cache client 104.
[0034] The illustrated example also shows requested data 118. The requested data 118 corresponds to data provided to the cache client 104 in response to the request 114. If the data identified in the request 114 is provided from the cache 102, e.g., upon a cache hit, the requested data 118 corresponds to cached data 116. However, in one or more scenarios, the data identified in the request 114 is provided from the data store 106. In a scenario where the data is not found in the cache 102 and is flagged as being stored in a non-cacheable location in the data store 106, for example, the requested data 118 corresponds to data provided to the cache client 104 from the data store 106. As described herein, a "cache miss" occurs when the controller 108 does not identify in the cache 102 the data identified by the request 114. A cache miss occurs, for example, if the data identified by the request 114 has not yet been loaded into the cache 102 or if the data identified by the request 114 has been evicted from the cache 102 prior to the request 114.
[0035] In various scenarios, the controller 108 loads data identified by the request from the data store 106 into the cache 102 in response to a cache miss. In the illustrated example, the data retrieved from the data store 106 and loaded into the cache 102 is shown as data store data 120. Once a cache miss is determined, for example, data requested by the request 114 is identified in the data store 106 and loaded from the data store 106 into one or more "locations" within the cache 102, for example, into one or more cache lines of the cache 102. This allows future requests for the same data to be served from the cache 102 rather than from the data store 106. As described in more detail below, the controller 108 loads data from the data store 106 (e.g., data store data 120) into the cache 102 based on at least one of the associativity locator 110 or the cache replacement policy 112.
[0036] In accordance with the described technology, the data store 106 is a computer-readable storage medium that stores data. Examples of the data store 106 include, but are not limited to, main memory (e.g., random access memory), external memory, a high-level cache (e.g., an L2 cache when the cache 102 is an L1 cache), secondary storage (e.g., mass storage devices), and removable media (e.g., flash drives, memory cards, compact discs, and digital video discs), to name a few. Examples of the data store 106 also include virtual memory that leverages the underlying secondary storage of a computing device according to one or more memory management techniques. It should be understood that the data store 106 can be configured in a variety of manners without departing from the spirit or scope of the described technology.
[0037] As described above, the controller 108 loads data from the data store 106 into the cache 102, for example, in response to a cache miss. In accordance with the described techniques, the controller 108 loads such data into the cache 102 based on at least one of an associativity allocator 110 or a cache replacement policy 112.
[0038] Cache replacement policy 112 controls which cache lines of cache 102 have their data evicted and loaded with data from data store 106 corresponding to request 114, e.g., in response to a cache miss. In one or more embodiments, cache replacement policy 112 is or includes a hardware maintained structure that manages the replacement of cache lines according to an underlying algorithm. Alternatively or additionally, cache replacement policy 112 is or includes a computer program that manages the replacement of cache lines according to an underlying algorithm. Exemplary cache replacement policies include, but are not limited to, first in first out, last in first out, least recently used, time-aware least recently used, most recently used, pseudo least recently used, random replacement, segmented least recently used, least-frequently used, least frequently recently used, and least frequently used with dynamic aging, to name a few. Exemplary embodiments in which the cache replacement policy 112 is configured at least in part according to a pseudo least recently used algorithm are described in more detail with respect to Figures 3 and 4.
[0039] The associativity allocator 110 limits which cache lines are available to different categories of cache requests that load data into the cache 102. In particular, the associativity allocator 110 allocates portions of the associativity of the cache 102 to different categories of cache requests, such as by allocating a first portion of the associativity to a first category of cache requests, a second portion of the associativity to a second category of cache requests, and so on. In one or more embodiments, the different categories correspond to different workloads or threads executed by the cache clients 104. For example, the first category is associated with requests corresponding to a first workload, and the second category is associated with requests corresponding to a second workload. Alternatively or additionally, categories are associated with requests based on different aspects, including, but not limited to, the request's origin or destination (e.g., the request originates from a particular computing unit or is sourced to a local scratch memory), the request characteristics (e.g., load, store, image sample, ray tracing, surface, buffer, or shader resource), memory request policy or coherency (e.g., streaming, locally cached, or globally coherent), and request age or forced forward progress flag (e.g., if a given request stream is stalled in an out-of-order cache for a certain amount of time due to an independent request stream, the given request stream can be isolated to ensure forward progress), to name a few. It should be appreciated that the associativity locator 110 allocates portions of the associativity of the cache 102 to different numbers of categories of requests in various embodiments. For example, in some variations, the associativity locator 110 allocates portions of the associativity to a single category of cache requests. In another example, the associativity allocator 110 allocates a portion of the associativity to two or more categories of cache requests.
[0040] The associativity locator 110 allocates a portion of the associativity of the cache 102 to a category of cache request by reserving a subset of the cache lines of the cache 102 for the category, such that data from the data store 106 corresponding to the category is loaded into the cache lines of the subset. Given an additional category of cache request, the associativity locator 110 allocates an additional portion of the associativity to the additional category by reserving an additional subset of the cache lines for the additional category. Data from the data store 106 corresponding to the additional category is loaded into the cache lines of this additional subset. Thus, for multiple categories of cache requests, the associativity locator 110 divides the associativity of the cache 102 into at least two portions, with each portion of the associativity corresponding to a respective subset of the cache lines of the cache 102.
[0041] In accordance with the described techniques, associativity defines a set of cache lines of cache 102 into which data at a location in data store 106 is permitted to be loaded, e.g., in response to a cache miss. In one or more embodiments, for example, cache 102 is fully associative, meaning that cache 102 permits data at a location in data store 106 to be loaded into any cache line of cache 102. Thus, in such embodiments, the set of cache lines corresponds to all cache lines of cache 102.
[0042] However, by allocating a portion of the associativity to a category of cache request, the associativity locator 110 further restricts which cache lines of the defined set of cache lines the data at a location in the data store 106 is permitted to be loaded into based on the category of the request associated with the data, e.g., the workload to which the request corresponds. Thus, the associativity locator 110 allows data at a location in the data store 106 to be loaded into a subset of the defined set of cache lines based on the category of the request. By further restricting which cache lines of the defined set data at different locations in the data store 106 is permitted to be loaded into based on the category, the associativity locator 110 prevents a particular category of cache request from dominating the use of all cache lines of the defined set that would otherwise be permitted to be used, e.g., taking into account the associativity of the cache 102. For example, when used in conjunction with an out-of-order cache, the associativity assigned by the associativity allocator 110 improves forward progress of requests, whereas in some prior art techniques, a request stream can dominate the out-of-order cache and starve other request streams. Allocating cache associativity as described above and below isolates the cache impact of multi-threading for deterministic behavior associated with tuning and debugging operations.
[0043] In one or more embodiments, the associativity locator 110 does not restrict which cache lines the controller 108 searches based on the request 114 to determine whether the data identified by the request 114 is stored in the cache 102, e.g., to detect a cache miss or a cache hit. Rather, the associativity locator 110 uses the cache replacement policy 112 to restrict which cache lines the controller 108 is permitted to evict data from and load data into in connection with a cache miss. For example, the controller 108 determines a category associated with the request 114. Depending on the portion of the associativity assigned to the category, the associativity locator 110 restricts which cache lines are available for allocation to the data corresponding to the request 114. In the context of assigning portions of the cache's associativity to categories of cache requests, consider the following description of FIG. 2.
[0044] 2 illustrates a non-limiting example 200 in which an associativity locator allocates portions of a cache's associativity to categories of cache requests. The example 200 includes the cache 102 and the associativity locator 110 of FIG.
[0045] In this example 200, cache 102 includes cache lines 202-216. Although cache 102 is shown as having eight cache lines in the illustrated example 200, it should be understood that cache 102 may include a different number of cache lines in various embodiments without departing from the described techniques.
[0046] Example 200 shows associativity locator 110 and cache 102 in a first stage 218 and a second stage 220, where the second stage 220 corresponds to a time after the time corresponding to the first stage 218. The first stage 218 shows cache 102 prior to when associativity locator 110 allocates a portion of the cache 102's associativity to a category of cache request.
[0047] The example 200 includes a trigger event 222 in the second phase 220. The trigger event 222 corresponds to at least one of a variety of events and triggers the associativity allocator 110 to allocate a portion of the associativity of the cache 102 to a category of cache requests. Examples of the trigger event 222 include, but are not limited to, launching an application and / or process for execution via the cache client 104, initializing or launching an additional workload or thread (e.g., while a workload or thread is executing via the cache client 104), determining that requests associated with a category of cache requests are dominating the usage of a defined set of cache lines (e.g., cache lines 202-216) such that performance associated with the additional category of cache requests is likely to be degraded as a result, determining that usage of the cache 102 exceeds a threshold usage (e.g., a usage frequency threshold, a threshold number of stalls, a threshold number of cache misses per time interval), determined real-time performance feedback (e.g., hit / miss ratio), or in response to a hardware event (e.g., a thrown exception), to name a few. Another example of triggering event 222 is software triggering that initiates software associativity allocation (e.g., directly from an application or based on compilation and / or driver feedback). In various embodiments, for example, software triggers associativity allocation for tuning and / or balancing, where associativity is allocated according to a programmed combination of categories for a single workload (or thread) or multiple workloads (or threads). Additional exemplary triggering events 222 include execution of unrelated workloads together (e.g., during virtualization when independent workloads share a single computing unit without knowledge of the other workloads) and receipt of a category (e.g., a "new" category) by controller 108.It should be appreciated that in various embodiments, different triggering events cause the associativity locator 110 to allocate portions of the associativity of the cache 102 to categories of cache requests.
[0048] Based on the trigger event 222, the associativity allocator 110 allocates the associativity of the cache 102, in part, by dividing the associativity into at least a first portion and a second portion. In this example 200, for example, the cache 102 is fully associative with respect to the set of cache lines 202-216. This means that the cache 102 allows data from a location in the data store 106 to be loaded into any of the cache lines 202-216 in a first stage 218, for example, in response to a cache miss. According to the described technique, the associativity allocator 110, based on a trigger event 222, divides the associativity of the cache 102 into a first portion corresponding to a first subset of the cache lines of the cache 102 (e.g., cache lines 202, 204, 206, 208) and a second portion corresponding to a second subset of the cache lines of the cache 102 (e.g., cache lines 210, 212, 214, 216).
[0049] When the associativity is split, the associativity allocator 110 allocates a first portion of the associativity to a first category of cache requests 224. As a result, rather than allowing the controller 108 to load data corresponding to the first category of cache requests 224 into any of the cache lines 202-216, as permitted by the associativity of the cache 102, the associativity allocator 110 restricts the controller 108 from loading such data into cache lines 202-208. In one or more embodiments, the associativity allocator 110 allocates a second portion of the associativity to a second category of cache requests (not shown). In such an embodiment, the associativity allocator 110 restricts the controller 108 from loading data corresponding to the second category of cache requests into cache lines 202-216, rather than allowing the controller 108 to load that data into any of cache lines 210-216 as permitted by the associativity of the cache 102.
[0050] In various embodiments, the associativity allocator 110 divides the associativity in different manners. For example, in at least one scenario involving two categories of cache requests (e.g., a first category of requests corresponding to a first workload or thread and a second category of requests corresponding to a second workload or thread), the associativity allocator 110 divides the associativity of the cache 102 into two portions, such as equal portions where the first category of requests are restricted to have their respective data loaded into one half of a cacheline and the second category of requests are restricted to have their respective data loaded into the other half of a cacheline.
[0051] In another example, the associativity allocator 110 divides the associativity of the cache 102 into four parts, such as in a scenario involving four categories of cache requests, e.g., a first category of requests corresponding to a first workload or thread, a second category of requests corresponding to a second workload or thread, a third category of requests corresponding to a third workload or thread, and a fourth category of requests corresponding to a fourth workload or thread. In at least one such scenario, the associativity allocator 110 divides the associativity evenly by restricting each category to loading its data into a respective quarter of a cache line. However, it should be appreciated that in one or more scenarios, the associativity allocator 110 does not divide the associativity evenly between the categories of cache requests, such as in at least one scenario involving three categories of cache requests. Although various divisions of associativity are described above, the associativity allocator 110 may be configured to divide and allocate the associativity of a set of cache lines in different manners without departing from the described techniques.
[0052] By allocating a portion of associativity to a category, the associativity allocator 110 limits which cache lines the cache replacement policy 112 is permitted to select to evict and load data corresponding to the category, e.g., in response to a cache miss. For example, for a given request, the cache replacement policy 112 is limited to selecting cache lines within the subset of cache lines reserved for the category corresponding to the request. In this regard, consider the following description of Figures 3 and 4.
[0053] FIG. 3 illustrates a non-limiting example 300 in which a cache replacement policy tree structure is traversed according to a cache replacement policy traversal algorithm to select a cache line.
[0054] The illustrated example 300 includes a tree structure 302. In this example 300, the tree structure is a binary tree. However, in other examples, the cache replacement policy 112 is implemented using other tree structures or without a tree structure. As illustrated, the tree structure 302 includes leaf nodes 304, 306, 308, 310. As described herein, a "leaf node" corresponds to a node in a binary tree that does not have child nodes. The leaf nodes 304, 306, 308, 310 correspond to cache lines of the cache 102. By way of example, the leaf node 304 corresponds to the cache line 202, the leaf node 306 corresponds to the cache line 204, the leaf node 308 corresponds to the cache line 206, and the leaf node 310 corresponds to the cache line 208. In scenarios where the traversal algorithm of the cache replacement policy 112 is not prevented from selecting any of the cache lines corresponding to the leaf nodes 304-310, such as because one or more cache lines are not replaceable or because of constraints on associativity, the traversal algorithm simply traverses the tree structure according to the respective rule sets to select a cache line for evicting and loading data.
[0055] In the illustrated example 300, a traversal algorithm is shown traversing a tree structure 302 in multiple stages, selecting cache lines for eviction and loading at each stage. The illustrated stages include a first stage 312, a second stage 314, a third stage 316, a fourth stage 318, a fifth stage 320, and a sixth stage 322. In particular, the multiple stages 312-322 illustrate traversal of the tree according to a pseudo least recently used algorithm, which is an example of a traversal algorithm for traversing a binary tree. As discussed above, in various embodiments, the cache replacement policy 112 is configured based on different algorithms without departing from the spirit or scope of the described technology.
[0056] According to the pseudo-least recently used method, each node of the tree structure 302 includes or is associated with a traversal direction indicator that indicates the direction of traversal of the tree structure from the node to its child nodes, e.g., the indicator indicates whether traversal proceeds from the node to its left child node or to its right child node. If a node is traversed during traversal, the directional indicator is switched to indicate the other direction for subsequent traversal, e.g., the node's indicator indicates to proceed to its left child node prior to traversal, and if the node is traversed during traversal (e.g., by proceeding from the node to its left child node as guided by the indicator), the indicator is switched to indicate to proceed from the node to its right child node when the node is next traversed, and vice versa.
[0057] In addition to leaf nodes 304-310, tree structure 302 in this example 300 also includes nodes 324, 326, and 328. Because tree structure 302 is a binary tree, each node (other than the leaf nodes) has two child nodes. In this example, nodes 324 and 326 are "child" nodes of node 328, and thus are "parents" of nodes 324 and 326. Similarly, leaf nodes 304 and 306 are "child" nodes of node 324 (and thus are "parents" of leaf nodes 304 and 306), and leaf nodes 308 and 310 are "child" nodes of node 326 (and thus are parents of leaf nodes 308 and 310).
[0058] In this example, each of the nodes 324, 326, 328 is shown with a graphical representation of a respective traversal direction indicator. In the first stage 312, for example, the graphical representation indicates that if node 328 is traversed, the traversal direction indicator of node 328 directs the algorithm to the left, if node 324 is traversed, the traversal direction indicator of node 324 directs the algorithm to the left, and if node 326 is traversed, the traversal direction indicator of node 326 directs the algorithm to the left. As mentioned above, according to the pseudo least recently used method, if a node is traversed during traversal, then the traversal direction indicator of the node is switched, for example, from pointing to the right for the left child node or from pointing to the left for the right child node.
[0059] To traverse the tree structure 302 according to the pseudo least recently used, the traversal algorithm of the cache replacement policy 112 starts from the root node, i.e., node 328. In the first phase 312, the traversal direction indicator of node 328 guides the traversal algorithm to proceed from node 328 to its left child node, i.e., node 324. Thus, the traversal algorithm traverses node 324. The traversal direction indicator of node 324 guides the traversal algorithm to proceed from node 324 to its left child node, i.e., leaf node 304, which corresponds to cache line 202 in this example. Since the child node of node 324 is a leaf node, the algorithm stops at leaf node 304 and therefore selects a cache line corresponding to leaf node 304 to evict the data and have new data loaded from data store 106. In the illustrated example 300, data store data 330 (graphically represented as "A") is thus loaded into cache line 202. The traversal direction indicators of the traversed nodes are switched in response to the traversal to evict data and load data store data 330. Because nodes 324, 328 are traversed to evict data and load data store data 330, the traversal indicators of these nodes are switched from leading the algorithm left to leading the algorithm right. Because node 326 is not traversed in the first stage 312, the traversal indicator of node 326 is not switched and remains leading the algorithm left.
[0060] Thus, in the second stage 314, the graphical representation of the respective traversal direction indicators indicates that if node 328 is traversed, the traversal direction indicator of node 328 directs the algorithm to the right, if node 324 is traversed, the traversal direction indicator of node 324 directs the algorithm to the right, and if node 326 is traversed, the traversal direction indicator of node 326 directs the algorithm to the left. In the second stage 314, the traversal algorithm of the cache replacement policy 112 starts traversing the tree structure 302 at the root node, i.e., node 328. In the second stage 314, the traversal direction indicator of node 328 directs the traversal algorithm to proceed from node 328 to its right child node, i.e., node 326. Thus, the traversal algorithm traverses node 326. The traversal direction indicator of node 326 directs the traversal algorithm to proceed from node 326 to its left child node, ie, leaf node 308, which in this example corresponds to cache line 206.
[0061] Because the child node of node 326 is a leaf node, the algorithm stops at leaf node 308 and therefore selects a cache line corresponding to leaf node 308 to have data evicted and new data loaded from data store 106. In the illustrated example 300, data store data 332 (graphically represented as "B") is therefore loaded into cache line 206. In response to the traversal to evict data and load data store data 332, the traversal direction indicators of the traversed nodes are switched. Because nodes 326, 328 are traversed to evict data and load data store data 332, the traversal direction indicators of these nodes are switched. Specifically, the traversal direction indicator of node 328 is switched from directing the algorithm to the right to directing the algorithm to the left, and the traversal direction indicator of node 326 is switched from directing the algorithm to the left to directing the algorithm to the right. Because node 324 is not traversed in the second stage 314, the traversal indicator for node 324 is not toggled, keeping the algorithm directing to the right.
[0062] Thus, in the third stage 316, the graphical representation indicates that if node 328 is traversed, the traversal direction indicator of node 328 directs the algorithm to the left, if node 324 is traversed, the traversal direction indicator of node 324 directs the algorithm to the right, and if node 326 is traversed, the traversal direction indicator of node 326 directs the algorithm to the right. In the third stage 316, the traversal algorithm of the cache replacement policy 112 begins traversing the tree structure 302 at the root node, i.e., node 328. In the third stage 316, the traversal direction indicator of node 328 directs the traversal algorithm to proceed from node 328 to its left child node, i.e., node 324. Thus, the traversal algorithm traverses node 324. The traversal direction indicator of node 324 directs the traversal algorithm to proceed from node 324 to its right child node, ie, leaf node 306, which in this example corresponds to cache line 204.
[0063] Because the child node of node 324 is a leaf node, the algorithm stops at leaf node 306 and therefore selects a cache line corresponding to leaf node 306 to have data evicted and new data loaded from data store 106. In the illustrated example 300, data store data 334 (graphically represented as "C") is therefore loaded into cache line 204. In response to the traversal to evict data and load data store data 334, the traversal direction indicators of the traversed nodes are switched. Because nodes 324, 328 are traversed to evict data and load data store data 334, the traversal direction indicators of these nodes are switched. Specifically, the traversal direction indicator of node 328 is switched from leading the algorithm left to leading the algorithm right, and the traversal direction indicator of node 324 is switched from leading the algorithm right to leading the algorithm left. Because node 326 is not traversed in the third stage 316, the traversal indicator for node 326 is not toggled, keeping the algorithm directing to the right.
[0064] The traversal algorithm of the cache replacement policy 112 continues to traverse the tree structure 302 of the cache replacement policy 112 according to the traversal direction indicator, switching the indicator of the traversed node over the fourth, fifth, and sixth stages 318, 320, and 322, respectively. In connection with such traversal, the cache replacement policy 112 further directs the eviction of data and the loading of data store data 336 (represented in the figure as "D") and data store data 338 (represented in the figure as "E") into cache lines corresponding to the illustrated leaf nodes. In the context of modifying the cache replacement policy 112 to assign it to different categories of cache requests, consider the following discussion.
[0065] 4 illustrates a non-limiting example 400 of an embodiment in which a cache replacement policy is modified to allocate a portion of the cache's associativity to categories of cache requests. In the illustrated example 400, the associativity locator 110 partially modifies the traversal algorithm of the cache replacement policy 112 to allocate a portion of the cache's 102 associativity to two categories of cache requests.
[0066] The illustrated example 400 includes a tree structure 402. In this example, the tree structure is a binary tree, although the cache replacement policy may be implemented using other tree structures or without a tree structure in various embodiments. As illustrated, the tree structure includes leaf nodes 404, 406, 408, 410, and nodes 412, 414, 416. Leaf nodes 404-410 correspond to cache lines of cache 102. By way of example, leaf node 404 corresponds to cache line 202, leaf node 406 corresponds to cache line 204, leaf node 408 corresponds to cache line 206, and leaf node 410 corresponds to cache line 208. Leaf nodes 404-410 are "child" nodes of node 412 and node 414. Furthermore, nodes 412 and 414 are "child" nodes of node 416, and are therefore "parents" of nodes 412 and 414.
[0067] In the example 400, the associativity locator 110 assigns associativity of the cache 102 to categories of cache requests by modifying the traversal algorithm used to traverse the tree structure 402 at a particular node of the tree structure 402. In this example, the associativity locator 110 modifies the traversal algorithm at node 416 but does not modify the traversal algorithm at other nodes of the tree structure 402. However, in variations, the associativity locator 110 modifies the traversal algorithm at more than one node or modifies the traversal algorithm in a different manner to assign associativity without departing from the spirit or scope of the described technology. For example, in some variations, the associativity locator 110 modifies the traversal algorithm at multiple nodes across the same level of the tree structure.
[0068] In this example 400, the traversal algorithm of the cache replacement policy 112 is pseudo least recently used, the details of which are described in more detail above in connection with Figure 3. The associativity locator 110 allocates a first portion of the cache 102's associativity to a first category of requests (e.g., corresponding to a first workload or thread) and allocates a second portion of the cache 102's associativity to a second category of requests (e.g., corresponding to a second workload or thread). To allocate the portions of the associativity to the categories, in this example 400, the associativity locator 110 modifies the traversal algorithm by "locking" or otherwise setting the traversal direction indicator of the node 416 to a first direction (e.g., left) for the first category of requests and a second direction (e.g., right) for the second category of requests.
[0069] By locking the traversal direction indicator of node 416 in different directions for the first and second categories of requests, the associativity locator 110 restricts the traversal algorithm of the cache replacement policy 112 to selecting the first category cache lines 202, 204 corresponding to the leaf nodes 404, 406, respectively. The associativity locator 110 further restricts the traversal algorithm to selecting the second category cache lines 206, 208 corresponding to the leaf nodes 408, 410, respectively. In this manner, the associativity locator reserves half of the associativity (corresponding to the cache lines 202 and 204) for the first category cache requests and the other half of the associativity (corresponding to the cache lines 206 and 208) for the second category cache requests.
[0070] To illustrate this, example 400 includes a first series of stages 418, 420, 422, where the traversal direction indicator of node 416 is locked to point left for requests corresponding to a first category, and example 400 also includes a second series of stages 424, 426, 428, where the traversal direction indicator of node 416 is locked to point right for requests corresponding to a second category.
[0071] In this example 400, data store data 430 (graphically represented as "A") and data store data 432 (graphically represented as "B") correspond to a first category of cache requests. For example, data store data 430 corresponds to a request to access cache 102 that resulted in a cache miss, the request being associated with a first category, such as a first workload or thread. Additionally, data store data 432 corresponds to an additional request to access cache 102 that resulted in a cache miss, the additional request also being associated with a first category, such as a first workload or thread.
[0072] In a first phase 418 of the first series, the cache replacement policy 112 begins traversing the tree structure 402 at the root node, i.e., node 416. Since the associativity locator 110 has modified the traversal algorithm at node 416 by locking its traversal direction indicator to the left for the first category, the traversal algorithm proceeds from node 416 to its left child node, i.e., node 412. Thus, the traversal algorithm traverses node 412. Since node 412 is not locked, node 412 is traversed according to the unmodified traversal algorithm with the pseudo least recently used used in this example. The traversal direction indicator of node 412 directs the traversal algorithm to proceed from node 412 to its left child node, i.e., leaf node 404, which corresponds to cache line 202 in this example. Because the child node of node 412 is a leaf node, the algorithm stops at leaf node 404 and therefore selects the cache line corresponding to leaf node 404 to evict the data and have new data loaded from data store 106 .
[0073] In the illustrated example 400, data store data 430 is therefore loaded into cache line 202. In response to the traversal to evict data and load data store data 430, the traversal direction indicators of the traversed nodes are switched according to the pseudo least recently used. Because node 412 is traversed to evict data and load data store data 430, the traversal indicator of node 412 is switched from directing the algorithm left to directing the algorithm right. Because node 416 is locked by associativity locator 110, the traversal direction indicator of node 416 is not switched, remaining left directing traversal associated with the first category of requests.
[0074] In the second stage 420 of the first series, the cache replacement policy 112 begins traversing the tree structure 402 at the root node, i.e., node 416. Since the traversal direction indicator of node 416 is locked to the left, the traversal algorithm proceeds from node 416 to its left child node, i.e., node 412. Thus, the traversal algorithm traverses node 412. As described above, node 412 is not locked and its traversal direction indicator is switched by the traversal in the first stage 418 to direct the algorithm to the right in the subsequent traversal. As a result, in the second stage 420, the traversal direction indicator of node 412 directs the traversal algorithm to proceed from node 412 to its right child node, i.e., leaf node 406, which corresponds to cache line 204 in this example. Because this child node of node 412 is a leaf node, the algorithm stops at leaf node 406 and therefore selects the cache line corresponding to leaf node 406 to evict the data and have new data loaded from data store 106 .
[0075] In the illustrated example 400, data store data 432 is therefore loaded into cache line 204. In response to the traversal to evict data and load data store data 432, the traversal direction indicator of the traversed node is switched according to the pseudo least recently used. Because node 412 is traversed to evict data and load data store data 432, the traversal indicator of node 412 is switched from directing the algorithm to the right to directing the algorithm to the left, as shown in the third stage 422 of the first series. Because node 416 is locked by associativity locator 110, the traversal direction indicator of node 416 is not switched and remains directing the traversal associated with the first category of requests to the left.
[0076] In contrast, data store data 434 (graphically represented as "C") and data store data 436 (graphically represented as "D") correspond to a second category of cache requests. For example, data store data 434 corresponds to a request to access cache 102 that resulted in a cache miss, the request being associated with a second category, such as a second workload or thread. Further, data store data 436 corresponds to an additional request to access cache 102 that resulted in a cache miss, the additional request being associated with a second category, such as a second workload or thread.
[0077] In the first phase 424 of the second series, the cache replacement policy 112 begins traversing the tree structure 402 at the root node, i.e., node 416. Since the associativity locator 110 has modified the traversal algorithm at node 416 by locking its traversal direction indicator to the right for the second category, the traversal algorithm proceeds from node 416 to its right child node, i.e., node 414. Thus, the traversal algorithm traverses node 414. Since node 414 is not locked, node 414 is traversed according to the unmodified traversal algorithm with the pseudo least recently used used in this example. The traversal direction indicator of node 414 directs the traversal algorithm to proceed from node 414 to its left child node, i.e., leaf node 408, which corresponds to cache line 206 in this example. Because the child node of node 414 is a leaf node, the algorithm stops at leaf node 408 and therefore selects the cache line corresponding to leaf node 408 to evict the data and have new data loaded from data store 106 .
[0078] In the illustrated example 400, data store data 434 is therefore loaded into cache line 206. In response to the traversal to evict data and load data store data 434, the traversal direction indicators of the traversed nodes are switched according to the pseudo least recently used. Because node 414 is traversed to evict data and load data store data 434, the traversal indicator of node 414 is switched from directing the algorithm left to directing the algorithm right. Because node 416 is locked by associativity locator 110, the traversal direction indicator of node 416 is not switched, leaving the traversal associated with the second category of requests directing right.
[0079] In the second stage 426 of the second series, the cache replacement policy 112 begins traversing the tree structure 402 at the root node, i.e., node 416. Since the traversal direction indicator of node 416 is locked to the right for the second category, the traversal algorithm proceeds from node 416 to its right child node, i.e., node 414. Thus, the traversal algorithm traverses node 414. As described above, node 414 is not locked and its traversal direction indicator is switched by the traversal in the first stage 424 to direct the algorithm to the right in the subsequent traversal. As a result, in the second stage 426, the traversal direction indicator of node 414 directs the traversal algorithm to proceed from node 414 to its right child node, i.e., leaf node 410, which corresponds to cache line 208 in this example. Because this child node of node 414 is a leaf node, the algorithm stops at leaf node 410 and therefore selects the cache line corresponding to leaf node 410 to evict the data and have new data loaded from data store 106 .
[0080] In the illustrated example 400, data store data 436 is therefore loaded into cache line 208. In response to the traversal to evict data and load data store data 436, the traversal direction indicator of the traversed node is switched according to the pseudo least recently used. Because node 414 is traversed to evict data and load data store data 436, the traversal indicator of node 414 is switched from directing the algorithm to the right to directing the algorithm to the left, as shown in the third stage 428 of the second series. Because node 416 is locked by associativity locator 110, the traversal direction indicator of node 416 is not switched and remains directing the traversal associated with the first category of requests to the left.
[0081] FIG. 5 illustrates a procedure 500 in an exemplary embodiment for allocating portions of a cache's associativity to categories of cache requests.
[0082] A portion of the cache's associativity is assigned to a category of the cache request (block 502). In accordance with principles described herein, the portion of the associativity corresponds to a subset of the cache lines of the cache. By way of example, the associativity allocator 110 assigns the portion of the cache's 102 associativity to a category of the cache request by reserving a subset of the cache lines of the cache 102 for the category such that data from the data store 106 corresponding to the category is loaded into the cache lines of the subset. By way of example, the associativity allocator 110 assigns a first portion of the cache's 102 associativity to a first category 224 of the cache request. The first portion of the cache's 102 associativity corresponds to, for example, cache lines 202-208. By allocating a first portion of the associativity of the cache 102 to the first category 224, the associativity allocator 110 restricts the controller 108 from loading data associated with a subsequent cache request via the first category 224 into the cache lines 202-208.
[0083] A request to access the cache is received (block 504), and it is determined that the request is associated with a category (block 506). As an example, the controller 108 receives a request 114 to access the cache 102, and the controller 108 determines a category associated with the request 114. For example, the controller 108 determines that the request 114 is associated with the first category 224.
[0084] A cacheline of the subset of cachelines is assigned to the request, and data corresponding to the request is loaded into the cacheline of the subset of cachelines (block 508). As an example, if the controller 108 determines that the request 114 is associated with the first category of cache requests 224, then the data store data 120 corresponding to the request 114 is loaded into one of the cachelines 202-208 of the cache 102 that is assigned to the first category of cache requests 224 by the associativity locator.
[0085] FIG. 6 illustrates a procedure 600 in an exemplary embodiment for partitioning cache associativity and allocating portions of the cache associativity to different categories of cache requests.
[0086] The associativity of the cache is divided into at least a first portion and a second portion (block 602). By way of example, the associativity allocator 110 divides the associativity of the cache 102 into a first portion corresponding to a first subset of the cache lines of the cache 102 (e.g., cache lines 202, 204, 206, 208) and a second portion corresponding to a second subset of the cache lines of the cache 102 (e.g., cache lines 210, 212, 214, 216).
[0087] A first portion of the cache's associativity is allocated to the first category of cache requests (block 604). In accordance with principles described herein, the allocating limits the first category of cache requests to loading data using the first portion of the cache's associativity in response to a cache miss. By way of example, when the associativity is split, the associativity allocator 110 allocates the first portion of the cache's 102 associativity to the first category of cache requests 224. As a result, rather than allowing the controller 108 to load data corresponding to the first category of cache requests 224 into any of the cache lines 202-216 as permitted by the associativity of the cache 102, the associativity allocator 110 limits the controller 108 to loading such data using the first portion of the cache's 102 associativity corresponding to cache lines 202, 204, 206, 208.
[0088] A second portion of the cache associativity is allocated to a second category of cache requests (block 606). In accordance with principles described herein, the allocating restricts the second category of cache requests to loading data using the second portion of the cache associativity in response to a cache miss. By way of example, the associativity allocator 110 allocates the second portion of the associativity to the second category of cache requests. In such an embodiment, the associativity allocator 110 restricts the controller 108 from loading data corresponding to the second category of cache requests into cache lines 202-216, rather than allowing the controller 108 to load that data into any of cache lines 210-216 as permitted by the associativity of the cache 102.
[0089] It should be understood that many variations are possible based on the disclosure herein, and although features and elements are described above in specific combinations, each feature or element can be used alone without the other features and elements, or in various combinations with other features and elements, with or without the other features and elements.
[0090] The various functional units illustrated in the figures and / or described herein (including, where appropriate, cache 102, cache client 104, data store 106, controller 108, associativity locator 110, and cache replacement policy 112) may be implemented in any of a variety of different manners, such as hardware circuits, software or firmware running on a programmable processor, or any combination of two or more of hardware, software, and firmware. The provided methods may be implemented in any of a variety of devices, such as a general purpose computer, a processor, or a processor core. Suitable processors include, by way of example, a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a graphics processing unit (GPU), a parallel accelerated processor, multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) circuit, any other type of integrated circuit (IC), and / or a state machine.
[0091] In one or more embodiments, the methods provided herein may be implemented in a computer program, software, or firmware embodied in a non-transitory computer-readable storage medium for execution by a general purpose computer or processor. Examples of non-transitory computer-readable storage media include read only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs).
[0092] (Conclusion) Although the systems and techniques have been described in language specific to structural features and / or methodological acts, it is to be understood that the systems and techniques defined in the appended claims are not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claimed invention.
Claims
1. 1. A method comprising: allocating a portion of the associativity of a cache to a category of cache requests, the portion of the associativity corresponding to a subset of cache lines of the cache; receiving a request to access the cache; assigning a cacheline of the subset of cachelines to the request based on a category associated with the request, and loading data corresponding to the request into a cacheline of the subset of cachelines. method.
2. allocating a portion of the associativity includes locking traversal of at least one node of a tree structure in a direction, the tree structure being traversed to identify which cache lines to allocate to the request, the traversal being locked in the direction for requests associated with the category.
2. The method of claim 1.
3. allocating additional portions of the associativity of the cache to additional categories of cache requests, the additional portions of the associativity being allocated by locking traversal of the at least one node of the tree structure in a different direction for requests associated with the additional categories. The method of claim 2.
4. the tree structure is a binary tree that is traversed according to a pseudo-least recently used algorithm to identify which cache line to allocate to the request. The method of claim 2.
5. allocating a cache line of the subset of cache lines; and loading data corresponding to the request by traversing the tree structure. The method of claim 2.
6. allocating additional portions of the associativity of the cache to additional categories of cache requests, the additional portions of the associativity corresponding to additional subsets of cache lines of the cache; receiving an additional request associated with the additional category to access the cache; allocating a cache line of the additional subset of cache lines to the additional request and loading additional data corresponding to the additional request into a cache line of the additional subset of cache lines.
2. The method of claim 1.
7. the category corresponds to a first workload or thread and the additional category corresponds to a second workload or thread. The method of claim 6.
8. determining that a category associated with the request corresponds to a category of the cache request; 2. The method of claim 1.
9. The categories of cache requests are: The workload or thread of the cache request; The origin of the cache request; the destination of the cache request, or Characteristics of the cache request is associated with at least one of 2. The method of claim 1.
10. Allocating a portion of the cache associativity to a category of cache requests occurs in response to a trigger event; The trigger event is Launching an application, Initializing a workload or thread; or determining when usage of the cache exceeds a threshold usage; Including any of the following:
2. The method of claim 1.
11. data corresponding to the request is retrieved from a data store; 2. The method of claim 1.
12. the data store includes a virtual memory; 12. The method of claim 11.
13. 1. A system comprising: a cache divided into cache lines; A controller, The controller: allocating a portion of the cache associativity to a category of cache requests, the portion of the cache associativity corresponding to a subset of the cache lines; allocating a cacheline of the subset of cachelines to the request based on a category of the request and loading data corresponding to the request into a cacheline of the subset of cachelines; 4. The method of claim 3, system.
14. the controller allocates a portion of the associativity of the cache by locking traversal of at least one node of a tree structure in a direction, the tree structure being traversed to identify which cache line to allocate to the request, the traversal being locked in the direction for requests associated with the category; The system of claim 13.
15. the controller is configured to allocate additional portions of the associativity of the cache to additional categories of cache requests, the additional portions of the associativity being allocated by locking traversal of the at least one node of the tree structure in a different direction for requests associated with the additional categories.
15. The system of claim 14.
16. the tree structure is a binary tree that is traversed according to a pseudo-least recently used algorithm to identify which cache line to allocate to the request.
16. The system of claim 15.
17. The cache is connected to an external memory. The system of claim 13.
18. The system includes a server, a personal computer, or a mobile device. The system of claim 13.
19. 1. A method comprising: Dividing the associativity of the cache into at least a first portion and a second portion; allocating a first portion of the cache associativity to a first category of cache requests, where in response to a cache miss, restricting the first category of cache requests to load data using the first portion of the cache associativity; allocating a second portion of the cache associativity to a second category of cache requests, and restricting the second category of cache requests to load data using the second portion of the cache associativity in response to a cache miss. method.
20. Allocating a first portion of the cache associativity to a first category of cache requests allows the first category of cache requests to access both the first and second portions of the cache associativity, but prevents the first category of cache requests from loading data using the second portion of the cache associativity in response to the cache miss; Allocating a second portion of the cache associativity to a second category of cache requests allows the second category of cache requests to access both the first and second portions of the cache associativity, but prevents the second category of cache requests from using the first portion of the cache associativity to load data in response to the cache miss.
20. The method of claim 19.
