Rereference interval prediction (RRIP) using pseudo-LRU acquisition timing information

Abstract

A system and method for cache replacement is disclosed. A technique is described for determining a Re-Reference Interval Prediction (RRIP) value for each data block in a cache, the RRIP value representing the likelihood that each data block will be reused within a time interval. A binary tree that tracks the freshness of data blocks in the cache is utilized to select a data block in the cache to be replaced by a data segment when the data segment is not stored in the cache upon access by a processor in memory.

Description

Technical Field

[0001] (Cross - reference to Related Applications) This application claims the benefit of U.S. Patent Application No. 17 / 489,726, filed on September 29, 2021, the content of which is incorporated herein by reference in its entirety as if fully set forth herein.

Background Art

[0002] Caching stores frequently accessed data in a memory (cache) local to the processor, thereby reducing the time it takes for the processor to access data in the main memory, and thus making access faster and more efficient. A cache replacement algorithm (or policy) is used by cache management software to determine which data block in the cache to evict when a processor memory access results in a miss. That is, when the processor needs to read from or write to a data segment in the main memory, if that data segment is already stored in the cache (i.e., a cache hit), the data segment is accessed from the cache. However, if that data segment is not yet stored in the cache (i.e., a cache miss), the data segment is fetched from the main memory and will replace a data block in the cache determined by the replacement algorithm.

[0003] A replacement policy predicts the likelihood that each data block in the cache will be rereferenced (reused), and based on this, selects to replace the data block that is predicted to be rereferenced in the far future. One example of a replacement policy is the Least Recently Used (LRU) replacement policy. The LRU replacement algorithm predicts that the data block that has not been used for the longest time is most likely to be rereferenced in the far future, and therefore selects that data block to be replaced. On the other hand, the Re-Reference Interval Prediction (RRIP) replacement algorithm considers the frequency at which a data block is rereferenced instead of recency, and predicts the time interval at which a data block is likely to be reused. Data blocks that are predicted to be rereferenced in the near immediate time interval are preserved, while data blocks that are predicted to be rereferenced in the distant time interval are candidates for eviction.

[0004] The replacement algorithms vary in part depending on what likelihood of re-referencing they assign to newly added data blocks (hit promotion policy), and how they update (learn) the likelihood of data blocks being reused. How well the replacement algorithms work depends on the data access patterns of the application running on the processor. For example, assigning near-immediate or far-fetched likelihoods to newly added data blocks is not optimal across all access patterns. Techniques for cache replacement that work well under different data access patterns are needed.

[0005] A more detailed understanding can be gained from the following explanation, which is provided as an example along with the attached diagram. [Brief explanation of the drawing]

[0006] [Figure 1A]This is a block diagram of an exemplary device in which one or more features of this disclosure may be implemented. [Figure 1B] This is a block diagram of an exemplary system that provides for managing memory access requests from a device in which one or more features of this disclosure may be implemented. [Figure 2] Figure 1A is a block diagram of an exemplary cache structure available to the processor of the device, on which one or more features of this disclosure may be implemented. [Figure 3] This is a functional block diagram of an exemplary cache replacement method based on a rereference interval prediction (RRIP) framework, on which one or more features of this disclosure may be implemented. [Figure 4] This is a block diagram of an exemplary tree-based pseudo-LRU (PLRU) selection procedure, on which one or more features of the present disclosure may be implemented. [Modes for carrying out the invention]

[0007] This disclosure describes a system and method for cache replacement that incorporates a tree-based pseudo-LRU (PLRU) algorithm into an RRIP framework. The RRIP algorithm described herein learns the re-reference likelihood of each data block in the cache. The RRIP algorithm maintains an RRIP value that can represent multiple re-reference time intervals. Thus, a data block can be predicted to be re-referenced at time intervals ranging from near-immediate (lowest RRIP value) through intermediate time intervals to distant time intervals (highest RRIP value). For example, when a new data block is placed in the cache, an intermediate re-reference interval can be assigned to it, and each time this data block is reused, a shorter time interval can be assigned to it, i.e., it is predicted to be re-referenced sooner. When a data block in the cache must be replaced (cache miss), a data block predicted to be re-referenced at a distant time interval may be selected for replacement. If no such data block is found in the cache, the RRIP value of the data block in the cache is incremented, which is a process that allows adaptation to changes in access patterns. However, if multiple data blocks with distant time interval predictions can be found in the cache, the PLRU algorithm's binary tree can be used to select the oldest data block, as described herein. Applying the PLRU algorithm in synergy with the RRIP algorithm results in a higher cache hit rate compared to applying the RRIP algorithm alone, given the cache access patterns of many workloads.

[0008] This disclosure describes a method for cache replacement. The method includes determining an RRIP value associated with each data block in the cache. The RRIP value represents the likelihood that the associated data block will be reused within a given time interval, with a larger RRIP value corresponding to a more distant time interval in the future. If a data segment is not stored in the cache when a processor accesses it in memory, the method includes selecting a data block in the cache to be replaced by the data segment. The selection of a data block includes searching for a data block in the cache associated with the RRIP value having the maximum value, and, if multiple data blocks are found, selecting one of the data blocks using a binary tree that tracks the relevance of data blocks in the cache.

[0009] This disclosure further discloses a system comprising at least one processor and memory. The memory stores instructions, which, when executed by at least one processor, cause the system to determine an RRIP value associated with each of the data blocks in the cache. The RRIP value represents the likelihood that the associated data block will be reused within a given time interval, with a larger RRIP value corresponding to a more distant time interval in the future. If the data segment is not stored in the cache when at least one processor accesses the data segment in memory, the instruction causes the system to select a data block in the cache to be replaced by the data segment. The selection of a data block includes searching for a data block in the cache associated with the RRIP value having the maximum value, and, if multiple data blocks are found, selecting one of the data blocks using a binary tree that tracks the relevance of the data blocks in the cache.

[0010] Furthermore, the disclosure discloses a non-transient computer-readable storage medium containing instructions executable by at least one processor for carrying out the method. The method includes determining a rereference interval prediction (RRIP) value associated with each data block in the cache. The RRIP value represents the likelihood that the associated data block will be reused within a time interval, with a larger RRIP value corresponding to a further time interval in the future. If the data segment is not stored in the cache when at least one processor accesses the data segment in the medium, the method includes selecting a data block in the cache to be replaced by the data segment. The selection of a data block includes searching for a data block in the cache associated with the RRIP value having the maximum value, and, if multiple data blocks are found, selecting one of the data blocks using a binary tree that tracks the relevance of data blocks in the cache.

[0011] Figure 1A is a block diagram of an exemplary device 100A in which one or more features of the present disclosure may be implemented. Device 100A may be, but is not limited to, a computer, a gaming device, a handheld device, a set-top box, a television, a mobile phone, a tablet computer, or any other computing device. Device 100A includes a processor 102, memory 104, storage device 106, one or more input devices 108, and one or more output devices 110. Device 100A also includes one or more input drivers 112 and one or more output drivers 114. Each input driver 112 may be embodied as hardware, a combination of hardware and software, or software, and may control the input device 108 (e.g., control its operation, receive input from the input driver 112, and provide data to the input driver 112). Similarly, each output driver 114 may be embodied as hardware, a combination of hardware and software, or software, and may play a role in controlling the output device 110 (for example, controlling its operation, receiving input from the output driver 114, and providing data to the output driver 114). It should be understood that device 100A may include additional components not shown in Figure 1A.

[0012] In various alternatives, the processor 102 includes a central processing unit (CPU), a graphics processing unit (GPU), a CPU and GPU located on the same die, or one or more processor cores, each of which can be a CPU or a GPU. In various alternatives, the memory 104 is located on the same die as the processor 102 or separately from the processor 102. The memory 104 includes volatile or non-volatile memory (e.g., random access memory (RAM), dynamic RAM, cache). In some embodiments, the processor 102 includes one or more caches 120. As is generally known, a cache stores data items that the cache manager has determined are likely to be accessed by instructions executed on the processor 102 in the near future. The caches 120 represent one or more cache memories of the processor 102. In various embodiments, one or more cache memories are organized into a cache hierarchy, where higher caches in the cache hierarchy serve misses from lower caches in the cache hierarchy and receive cache entries that have been missed from lower caches in the cache hierarchy.

[0013] The storage device 106 includes fixed or removable storage devices (e.g., hard disk drives, solid-state drives, optical discs, flash drives, etc.). The input device 108 includes, but is not limited to, a keyboard, keypad, touchscreen, touchpad, detector, microphone, accelerometer, gyroscope, biometric scanner, or network connectivity (e.g., a wireless local area network card for transmitting and / or receiving wireless IEEE 802 signals). The output device 110 includes, but is not limited to, a display, speaker, printer, haptic feedback device, one or more lights, antennas, or network connectivity (e.g., a wireless local area network card for transmitting and / or receiving wireless IEEE 802 signals).

[0014] The input driver 112 and the output driver 114 each include one or more hardware, software, and / or firmware components configured to interface with and drive the input device 108 and the output device 110, respectively. The input driver 112 communicates with the processor 102 and the input device 108, enabling the processor 102 to receive input from the input device 108. The output driver 114 communicates with the processor 102 and the output device 110, enabling the processor 102 to send output to the output device 110. The output driver 114 includes an accelerated processing device (APD) 116 coupled to the display device 118, which in some examples is a physical display device or a simulated device that displays output using a remote display protocol. The APD 116 is configured to accept computation and graphics rendering commands from the processor 102, process those computation and graphics rendering commands, and provide pixel output to the display device 118 for display. As will be described in more detail below, the APD116 includes one or more parallel processing units configured to perform computations according to the single-instruction-multiple-data (SIMD) paradigm. Thus, although various functions are described herein as being performed by or in conjunction with the APD116, in various alternative examples, functions described as being performed by the APD116 are not driven by the host processor (e.g., processor 102) and are additionally or alternatively performed by other computing devices having similar capabilities configured to provide graphic output to the display device 118. For example, any processing system that performs processing tasks according to the SIMD paradigm is intended to be configured to perform the functions described herein. Alternatively, a computing system that does not perform processing tasks according to the SIMD paradigm is intended to perform the functions described herein.

[0015] Figure 1B is a block diagram of an exemplary system 100B that provides for managing memory access requests from a device in which one or more features of the present disclosure may be implemented. System 100B includes a processor 130 and memory 122. System 100B can be incorporated into any of a variety of electronic devices, such as a server, personal computer, tablet, set-top box, or gaming system. The processor 130 is generally configured to execute a set of instructions (e.g., a computer program) that operate the circuitry of the processor 130 to perform a defined task.

[0016] Memory 122 facilitates the execution of tasks by storing data and instructions used by the processor 130. Memory 122 may be random access memory (RAM), non-volatile memory (e.g., flash memory or hard disk drive (HDD)), or a combination thereof. System 100B also includes devices 124 such as physical input / output (I / O) devices. Devices 124 may be, for example, a network interface card (NIC), a host bus adapter (HBA), etc.

[0017] The processor 30 also includes processor cores 140, 150, 160, and 170, and a memory controller (e.g., a northbridge) 132. The processor cores 140, 150, 160, and 170 are processing units that execute instructions individually and in parallel. In some embodiments, each of the processor cores 140, 150, 160, and 170 includes an individual instruction pipeline that fetches instructions, decodes the fetched instructions into corresponding operations, and performs various operations using the resources of the system 100B. Figure 1B shows a processor 130 having four processor cores 140, 150, 160, and 170, but the processor 130 may have more or fewer processor cores than four. Each of the processor cores 140, 150, 160, and 170 has a low-level cache dedicated to that processor core. For example, processor core 140 has a level 1 cache 142 and a level 2 cache 144. Furthermore, processor core 140 may include a translation index buffer 146. Similarly, processor core 150 has associated level 1 caches 152 and level 2 caches 154. Processor core 150 may also include a translation index buffer 156. In a further example, processor core 160 has level 1 caches 162 and level 2 caches 164. Processor core 160 may also include a translation index buffer 166. In another example, processor core 170 has level 1 caches 172 and level 2 caches 174. Processor core 170 may also include a translation index buffer 176. Processor 130 may further include cache memory shared by a pair of processors. For example, processor cores 140 and 160 may share an L3 cache memory 180, and processor cores 150 and 170 may share an L3 cache memory 185.

[0018] The processor 130 also includes an input / output memory management unit (IOMMU) 136 used to connect a device (e.g., device 124 as shown in Figure 1A) to the memory controller 132. In some embodiments, the IOMMU 136 communicates with device 124 using a serial bus. The serial bus may conform to a bus standard such as the Peripheral Component Interconnect Express (PCIe) serial bus standard. The memory controller 132 provides an interface for device 124 to communicate with memory 122 or one or more levels of cache memory. The IOMMU 136 receives memory access requests from device 124 (e.g., direct memory access requests such as cache injection memory access requests) and controls the provision of those requests to memory 122 or cache memory via the memory controller 132. The processor 130 includes, for example, a physical tag map 134 in the memory controller 132 for mapping physical steering tags received from the IOMMU 136 to physical resources targeted by related memory access requests, such as cache injection memory access requests. In addition, the memory controller 132 receives responses to memory access requests from the memory 122 or cache memory and controls the provision of responses to the device 124.

[0019] Cache-injection memory access requests can use steering tags to target processor resources such as lower-level caches dedicated to a processor core, caches shared among subsets of processor cores, or shared caches. For example, a steering tag can target an L1 cache 152 or L2 cache 154 accessible by processor core 150, or an L3 cache 185 accessible by processor cores 150 and 170. Therefore, to facilitate the efficient execution of memory access requests, the IOMMU 136 accesses a virtual tag map to translate virtual steering tags received in a memory access request into physical steering tags mapped to physical resources. Upon receiving a memory access request from device 124, the IOMMU 136 identifies the virtual steering tag and translates it using the virtual tag map 138. The virtual tag map 138 can be stored in system memory 122, and a portion can be cached in the IOMMU 136. Alternatively, the virtual tag map 138 can be cached in the IOMMU 136. The IOMMU136 provides the memory access request and physical steering tag to the memory controller 132. The memory controller 132 translates the physical steering tag into the physical resource targeted by the memory access request, such as a lower-level cache targeted by the cache injection memory access request. In one example, the memory access request targets a low-level cache dedicated to the processor core. In such an example, the address associated with the request is translated by a translation index buffer, and a cache injection memory access may be performed.

[0020] In some embodiments, device 124 accesses information in memory 122 or cache memory through a direct memory access (DMA) request by initiating a memory access request that includes a virtual steering tag and a virtual address, which are sent to the IOMMU 136 using the memory controller 132. The IOMMU 136 can perform steering tag remapping between the virtual steering tag and the physical steering tag, and address translation between the virtual address and the physical address in memory 122 or cache memory. The steering tag remapping process and the address translation process may be independent functions of the IOMMU and may be performed simultaneously. For example, device 124 generates a cache injection memory access request that includes a write request to store data in cache memory associated with a processor running a virtual machine that has use for the data provided by device 124. Each memory access request includes a virtual steering tag mapped to a cache resource (e.g., L3 cache 180 or 185; L1 cache 142, 152, 162 or 172; L2 cache 144, 154, 164 or 174) through an associated physical steering tag, and a memory address indicating the location in the cache resource targeted by the cache injection memory access request. In response to a write request, the resource stores the write information for the request at the location corresponding to the memory address of the write request.

[0021] Figure 2 is a block diagram of an exemplary cache structure 200 available to the processor of the device in Figure 1A, on which one or more features of this disclosure may be implemented. The cache 210 includes data blocks 220.1 to 220.N (also called “lines”). Each data block may contain data segments previously fetched from memory. For example, line 220.n may contain data segment 250 fetched from address 255 in main memory 240. In addition to the data blocks, each line in the cache (e.g., 220.n) is associated with a tag (e.g., 215.n) and a flag bit (e.g., 225.n). For example, tag 215.n is associated with address 255 in main memory 240 from which data segment 250 (currently stored in line 220.n) was fetched. Flag bits 225.n are typically used to store data that characterizes (or is associated with) the corresponding data block 220.n. For example, flag 225.n can indicate the validity of the data stored in data block 220.n, or whether main memory has not been updated as a result of changes made to the data stored in data block 220.n.

[0022] Therefore, tags 215.1 to 215.N are associated with each line 220.1 to 220.N in the cache 210, each representing an actual or virtual address in main memory 240. When the processor executes an instruction that requires memory access, i.e., when the processor needs to read from or write to a data segment stored at a specific address in main memory, the cache is first checked to see if the requested data segment is already stored in one of the cache lines 220.1 to 220.N. To determine whether the requested data segment 250 is already stored in one of the cache lines, the address 255 of the data segment can be compared to the tags 215.1 to 215.N of each line 220.1 to 220.N. However, in practice, the cache placement policy stores memory sections of main memory (such as data segment 250) at specific locations within the cache. Therefore, a cache is a fully associative cache if any line in the cache can store any copy of a memory section. A cache is a directly mapped cache if each memory section can be stored in only one line in the cache. A cache is an L-wayset associative cache if, as is the case normally, each memory section can be stored in a set of L locations in the cache.

[0023] Typically, the bits representing memory address 255 can be divided into groups based on their bit significance. In FIG. 2, memory address 230 is shown to include a tag 230.1, an index 230.2, and a block offset 230.3. The block offset 230.3, which has the least significant bits of the address 230, provides an offset indicating the position within the data block where the requested data segment 250 is stored. For example, if the data block is a b - byte block, the block offset 230.3 would have to be log2(b) bits long. The index 230.2 specifies a set of data blocks within the cache in which a particular memory section can be stored (e.g., one set contains L data blocks in an L - way set - associative cache). For example, if there are s sets, the index 230.2 would have to be log2(s) bits long. The tag 230.1, which has the most significant bits of the address 230, associates a particular memory section with a line (specified by the index) within the set in which it is stored. Thus, when searching whether a data segment is stored in the cache, the index part of the address of the data segment indicates the set within the cache in which the data segment can be stored, and then the tag part of the address of the data segment is compared with the tags of the lines within that set. If cache 210 is a 4 - way set - associative cache, the cache is divided into sets (specified by the index), and each set contains 4 ways (4 data blocks or lines). If the index part of the address 255 of data segment 250 indicates the first set, data segment 250 should be searched within this set, and by comparing the tag part of the address 255 of the data segment with the tags of the lines within that set, it can be found whether any of those lines stores the data segment.

[0024] Figure 3 is a functional block diagram of an exemplary cache replacement method 300 based on the RRIP framework, on which one or more features of this disclosure may be implemented. When the processor executes an instruction involving memory access (e.g., memory read or memory write), in step 310 the cache is checked to determine whether the data segment to be accessed (i.e., the requested data segment) is already stored in the cache's data block. The check to determine whether the requested data segment is already stored in the cache's data block may be performed, for example, as described above with reference to Figure 2. If the requested data segment is already stored in the cache's data block, a cache hit occurs in step 315, and therefore, in step 320, the data segment can be read from the data block. Following the rereference to the data block 320, in step 325, the rereference interval forecast associated with the data block may be updated according to the hit promotion policy. If the requested data segment is not stored in any of the cache's data blocks, a cache miss occurs in step 315, and therefore one of the cache's data blocks must be evicted (for example, if there are no invalid blocks available to use), i.e., the requested data segment is fetched from main memory and copied by the replacement algorithm 300 to the data block selected to be replaced.

[0025] Various cache replacement algorithms apply different replacement policies. In the RRIP framework for cache replacement, the data blocks least likely to be referenced will be evicted. For example, the RRIP framework can retain M bits associated with each data block in the cache, and 2 M A value of -1 can be represented. Therefore, a value of zero RRIP indicates that the data block is expected to be rereferenced in the near immediate future, and 2 MThe RRIP value of -1 indicates that the data block is predicted to be re-referenced in the distant future. For example, when M = 2, four RRIP values [0, 1, 2, 3] are possible, and the lower the RRIP value, the earlier each data block is predicted to be re-referenced.

[0026] Therefore, when a cache miss occurs, at step 330, the data block with the maximum RRIP value (the data block predicted to be referenced later than all other data blocks) is searched for. In one aspect, data blocks having RRIP values within a range including the maximum RRIP value may be searched for. If no data block with the maximum RRIP value (or within the range including the maximum RRIP value) is found, at step 340, the RRIP values of all data blocks are incremented by 1, and another search occurs at step 330. Otherwise, at step 345, if two or more data blocks with the maximum RRIP value (or within the range including the maximum RRIP value) are found, at step 355, one data block is selected to be replaced. If exactly one data block with the maximum RRIP value (or within the range including the maximum RRIP value) is found, at step 350, that block is replaced. Therefore, if two or more data blocks with the maximum RRIP value (or two or more data blocks having RRIP values within the range including the maximum RRIP value) are found, at step 355, a procedure for selecting one data block to be replaced must be performed. A simple selection procedure that can be used is, for example, to select the data block with the minimum index. As disclosed herein, a tree-based PLRU selection procedure can be used as will be described in detail below.

[0027] After it is determined which data block will be replaced, in step 350, the requested data segment is fetched from main memory and stored in that data block. For example, if data block 220.n is the one to be replaced, the requested data segment 250 is copied to data block 220.n, address 215.n is updated to address 255 of data segment 250 in main memory 240, and, if necessary, one or more flag bits are updated to reflect, for example, that the data block content is valid. Next, in step 360, the RRIP value of the replaced data block may be set depending on how soon the replaced data block 220.n is expected to be rereferenced. A reasonable rereference interval prediction depends on the memory access patterns of the processor (or the application the processor runs on). The replacement algorithm can learn possible rereference intervals, for example, by assigning a high RRIP value in step 360 when a data block is replaced, and then in step 320, decreasing this RRIP value by 1 each time this data block is rereferenced.

[0028] Figure 4 is a block diagram 400 of an exemplary tree-based PLRU selection procedure, on which one or more features of the present disclosure may be implemented. PLRU is a replacement algorithm that approximates the LRU replacement algorithm. While the LRU algorithm essentially replaces the data block with the oldest access time (the data block that has not been accessed for the longest time), the PLRU algorithm provides a result that approximates the data block with the oldest access time. The tree-based PLRU algorithm operates with a slightly higher miss rate compared to the LRU algorithm, but has better latency, uses less power, and has lower overhead. The tree-based PLRU maintains a binary tree that points to the next data block to be replaced. Thus, the binary tree 410 of the PLRU algorithm maintains a path to the data block with the approximate oldest access time.

[0029] As clearly shown in Figure 4, in the current state of the tree, data block 420.6 is pointed to be replaced via a tree path containing nodes 410.1, 410.3, and 410.6 in the binary tree. Generally, nodes with a value of "1" point to (or are connected to via tree branches) lower nodes to the right, and nodes with a value of "0" point to (or are connected to via tree branches) lower nodes to the left. Each time a data block such as data block 420.6 is replaced, the nodes connected to it (410.1, 410.3, and 410.6) are flipped (their values ​​are changed so that they point to lower nodes), and as a result, another data block in the cache is pointed to, in this case, as clearly shown by tree 440 pointing to data block 420.2, which is the next data block to be replaced.

[0030] As described above with reference to Figure 3, if two or more data blocks have the maximum RRIP value, a selection procedure can be applied to select one data block to be replaced from among them (step 355). The selection procedure disclosed herein can utilize a binary tree of a tree-based PLRU algorithm. Figure 4 illustrates a cache 420 having eight data blocks 420.1 to 420.8 and their respective RRIP values ​​430. As shown, three data blocks (420.1, 420.3, and 420.6) have the maximum RRIP value of 3. Therefore, a selection procedure can be used to select one data block based on the current state of the binary tree 410. As described above, the binary tree 410 points to data block 420.6. Since that block is one of the three blocks with the maximum RRIP value, that block will be selected to be replaced. Otherwise, for example, in a scenario where data block 420.6 has an RRIP value of 2 and only two blocks 420.1 and 420.3 have a maximum RRIP value of 3, the current state tree does not point to any of the data blocks with the maximum RRIP value, so the selection procedure can start from a lower level of the tree. For example, the selection procedure can start from node 410.2 and trace a path to the data block with the maximum RRIP value, in this case data block 420.3 (via node 410.5).

[0031] In an alternative, the selection procedure can begin at the lowest level of the tree (e.g., 410.4, 410.5, 410.6, and 410.7). If multiple blocks with the maximum RRIP value are pointed to at that level (e.g., 420.1, 420.3, and 420.6), the procedure can proceed to the next level (e.g., 410.2 and 410.3), and if multiple blocks with the maximum RRIP value are still pointed to at that level (e.g., 420.3 and 420.6), the procedure can proceed to higher levels of the tree until only one block is pointed to. At the highest level of the tree (e.g., 410.1), only one block 420.6 may be pointed to, and that block may be selected to be replaced. In one embodiment, data blocks 420.1, 420.3, and 420.6 can be associated with RRIP values ​​within a range that includes the maximum RRIP value. In such cases, the selection of one data block from them can be carried out as if they all associated the maximum RRIP value with them, as described above.

[0032] It should be understood that many modifications are possible based on the disclosures herein. Although features and elements are described above in specific combinations, each feature or element can be used alone without other features and elements, or in various combinations with or without other features and elements.

[0033] Various functional units shown in the figures and / or described herein (including, but not limited to, the processor 102, input driver 112, input device 108, output driver 114, output device 110, and accelerated processing device 116) may be implemented as a general-purpose computer, processor, or processor core, or as a program, software, or firmware stored on a non-temporary computer-readable storage medium or another storage medium executable by a general-purpose computer, processor, or processor core. The methods provided can be implemented in a general-purpose computer, processor, or processor core. Suitable processors include, by example, general-purpose processors, dedicated processors, conventional processors, digital signal processors (DSPs), multiple microprocessors, one or more microprocessors associated with a DSP core, controllers, microcontrollers, application-specific integrated circuits (ASICs), field-programmable gate array (FPGA) circuits, any other type of integrated circuit (IC), and / or state machines. Such a processor can be manufactured by configuring a manufacturing process using the results of processed hardware description language (HDL) instructions and other intermediate data (instructions that can be stored in a computer-readable medium), including a netlist. The results of such processing may be a mask work, which is used in a subsequent semiconductor manufacturing process to manufacture a processor that implements the features of the present disclosure.

[0034] The methods or flowcharts provided herein can be implemented in computer programs, software, or firmware embedded in a non-temporary computer-readable storage medium for execution by a general-purpose computer or processor. Examples of non-temporary computer-readable storage media include read-only memory (ROM), random-access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs).

Claims

1. A method for cache replacement, The cache controller determines the rereference interval prediction (RRIP) value associated with each data block in the cache, The processor accesses a data segment in memory, The process includes selecting a data block in the cache to be replaced by the data segment, depending on whether the data segment is not stored in the cache. Selecting the aforementioned data block means Searching for the data block in the cache associated with the RRIP value having the maximum value, If multiple data blocks are found, the process includes using a binary tree to select one of the multiple data blocks, method.

2. Using the aforementioned binary tree, selecting any of the multiple data blocks is: This includes selecting a data block having the maximum RRIP value that is uniquely pointed to from one level of the binary tree, The method according to claim 1.

3. The RRIP value represents the likelihood that the associated data block will be reused within a time interval. The method according to claim 1.

4. A higher RRIP value corresponds to a time interval in the far future. The method according to claim 1.

5. The aforementioned maximum RRIP value is a range of RRIP values ​​that includes the aforementioned maximum RRIP value. The method according to claim 1.

6. The aforementioned search is, If a data block associated with the RRIP value that has the maximum value is not found, Increasing the RRIP value associated with each data block in the cache, This includes repeating the aforementioned search, The method according to claim 1.

7. The aforementioned search is, If a data block associated with an RRIP value having the maximum value is found, the process includes selecting the said data block. The method according to claim 1.

8. When the processor accesses a data segment, if the data segment is stored in a data block in the cache, the RRIP value associated with the data block is updated. The method according to claim 1.

9. The binary tree tracks the up-to-dateness of the data blocks in the cache. The method according to claim 1.

10. It is a system, At least one processor, It includes a memory for storing instructions, When the instruction is executed by the at least one processor, The cache controller determines the rereference interval prediction (RRIP) value associated with each data block in the cache, The at least one processor accesses the data segment in the memory, Depending on whether the data segment is not stored in the cache, select a data block in the cache to be replaced by the data segment, The system performs this task, Selecting the aforementioned data block means Searching for the data block in the cache associated with the RRIP value having the maximum value, If multiple data blocks are found, the process includes selecting one of the multiple data blocks using a binary tree that tracks the relevance of the data blocks in the cache, system.

11. Using the aforementioned binary tree, selecting any of the multiple data blocks is: This includes selecting a data block having the maximum RRIP value that is uniquely pointed to from one level of the binary tree, The system according to claim 10.

12. The RRIP value represents the likelihood that the associated data block will be reused within a time interval. The system according to claim 10.

13. A higher RRIP value corresponds to a time interval in the far future. The system according to claim 10.

14. The maximum RRIP value is the range of RRIP values ​​that includes the aforementioned maximum RRIP value. The system according to claim 10.

15. The aforementioned search is, If a data block associated with the RRIP value that has the maximum value is not found, Increasing the RRIP value associated with each data block in the cache, This includes repeating the aforementioned search, The system according to claim 10.

16. The aforementioned search is, If a data block associated with an RRIP value having the maximum value is found, the process includes selecting the said data block. The system according to claim 10.

17. When at least one processor accesses a data segment, if the data segment is stored in a data block in the cache, an instruction is provided to cause the system to update the RRIP value associated with the data block, The system according to claim 10.

18. A computer-readable storage medium containing instructions that can be executed by at least one processor in order to perform a method, The aforementioned method, The cache controller determines a rereference interval prediction (RRIP) value associated with each data block in the cache, where the RRIP value represents the likelihood that the associated data block will be reused within a given time interval, with a higher RRIP value corresponding to a more distant future time interval. The aforementioned at least one processor accesses a data segment in memory, The process includes selecting a data block in the cache to be replaced by the data segment, depending on whether the data segment is not stored in the cache. Selecting the aforementioned data block means Searching for the data block in the cache associated with the RRIP value having the maximum value, If multiple data blocks are found, the process includes selecting one of the multiple data blocks using a binary tree that tracks the relevance of the data blocks in the cache, Computer-readable storage medium.

19. The aforementioned search is, If a data block associated with the RRIP value that has the maximum value is not found, Increasing the RRIP value associated with each data block in the cache, This includes repeating the aforementioned search, A computer-readable storage medium according to claim 18.

20. The aforementioned search is, If a data block associated with an RRIP value having the maximum value is found, the process includes selecting the said data block. A computer-readable storage medium according to claim 18.

21. When the processor accesses a data segment, if the data segment is stored in a data block in the cache, the RRIP value associated with the data block is updated. A computer-readable storage medium according to claim 18.

Images