Dynamic distance adjustment for stride prefetcher

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for dynamic distance adjustment for stride prefetcher. One of the systems includes a processor; a stride prefetcher; and a cache subsystem, wherein the stride prefetcher is configured to dynamically compute a stride prefetch distance based on a measure of cache miss latency and a duration between requests in a sequence of strided memory accesses issued by the processor.

Description

[0001] DYNAMIC DISTA CE ADJUSTMENT FOR STRIDE PREFETCHER

[0002] BACKGROUND

[0003]

[0001] This specification is related to systems with integrated circuit devices.

[0004]

[0002] Prefetching is a predictive technique used by processors to boost execution performance by fetching data before the data is requested. The data is typically fetched from a slower memory to a faster local memory, e.g., from random access memory to a cache, or from a higher-level cache (e.g., L2 or L3 cache) to a lower-level cache (e.g., LI cache). Prefetching data and then accessing the data from caches is usually much faster than retrieving the data when it is requested.

[0005]

[0003] A stride prefetcher can perform prefetching using a fixed address gap, which may be referred to as the stride. The stride means that accesses to the slower memory can be performed using a fixed address gap.

[0006]

[0004] Typically, due to memory latency, the cache is unable to prefetch the first element in a strided pattern and must therefore start prefetching from a subsequent element in a sequence. The distance between the first element in the sequence and the first prefetched element is referred to as the stride prefetch distance.

[0007]

[0005] The stride prefetch distance helps a stride prefetcher to prefetch data in a way that is not too early and not too late. A late prefetch occurs when the prefetched cache line cannot service the request on time. An early prefetch occurs when the prefetched cache line gets evicted from the cache before it can be used.

[0008] SUMMARY

[0009]

[0006] This specification describes techniques for dynamically adjusting a stride prefetch distance for a stride prefetcher based on a runtime estimate of the cache miss latency and a duration between requests. These techniques allow a stride prefetcher to generate a larger number of timely prefetches.

[0010]

[0007] Particular embodiments of the subject matter described in this specification can be implemented so as to realize one or more of the following advantages. The systems and methods can provide a low-cost tracking mechanism to determine the stride prefetch distance. The stride prefetcher can dynamically detect the cache miss latency and the duration between requests, and can use the cache miss latency and the duration between requests to dynamically adjust the stride prefetch distance. The stride prefetcher can generate timely prefetches and the prefetched lines can arrive at cache on time to service the requests. The systems and methods can provide online precise distance control without much cost on tracking every cache access behavior because the system can track the stride stream pattern and can sample part of a miss status holding register (MSHR) to get the cache miss latency from cache side. In some implementations, the system can further reduce the cycle time cost by using the hit level hint information from lower-level memory’, thus reducing the tracking cost from the MSHR.

[0011]

[0008] The details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subj ect matter will become apparent from the description, the drawings, and the claims.

[0012] BRIEF DESCRIPTION OF THE DRAWINGS

[0013]

[0009] FIG. 1 is a diagram of an example system.

[0014]

[0010] FIG. 2 is a diagram of an example stride pattern.

[0015] [OH] FIG. 3 is a flowchart of an example process for stride prefetching using a dynamically computed stride prefetch distance.

[0016]

[0012] FIG. 4A is an example table for a stride prefetcher with a dynamically computed stride prefetch distance.

[0017]

[0013] FIG. 4B is an example for additional information in a miss status holding register (MSHR) and cache for dynamic distance adjustment.

[0018]

[0014] FIG. 5A is a diagram of a miss status holding register (MSHR).

[0019]

[0015] FIG. 5B is a diagram of hit level hint information added in a data return packet.

[0016] Like reference numbers and designations in the various drawings indicate like elements.

[0020] DETAILED DESCRIPTION

[0021]

[0017] FIG. 1 is a diagram of an example system 100. The system 100 includes a processor 104 that provides memory' requests for locations in a memory' device 116.

[0018] The system 100 includes a stride prefetcher 112. The stride prefetcher 112 can detect a stride pattern of memory accesses and can perform prefetching using the stride pattern. The stride pattern means that accesses to the slower memory can have a fixed address gap, e.g., a stride. For example, a stream of accesses to the memory 116 can have the following addresses, A+k, A+2*k, A+3*k, ... , etc. In this example, the stride is k.

[0019] The stride prefetcher 112 can dynamically determine a stride prefetch distance during runtime, e.g., a number of strides the system should initially skip in order for the first prefetched data element to arrive in time at the cache. For example, when the stride prefetch distance is 2, the system can initially prefetch data at the address A+3*k. The prefetcher will thus not attempt to prefetch the addresses A+k and A+2*k because they likely would not arrive in time.

[0022]

[0020] Some systems determine a fixed stride prefetch distance, e.g., by computing (cache access latency of next level) / (cache access latency of current level). For example, if the LI access latency is 3 and the L2 access latency is 8, the stride prefetch distance can be calculated as a fixed number 8 / 3-2.67 ~ 3. However, the latency of each cache access is not fixed, and it can be dynamically changing depending on the different workload.

[0023]

[0021] The stride prefetcher 112 of the system 100 can dynamically compute a stride prefetch distance at runtime, instead of using a fixed distance. The stride prefetcher 112 can dynamically compute the stride prefetch distance based on a measure of cache miss latency and a duration between requests in a sequence of strided memory accesses issued by the processor 104.

[0024]

[0022] The measure of the cache miss latency can indicate the latency of bringing back a prefetched cache line. In some implementations, the measure of the cache miss latency can be the average cache miss latency from the current cache level. For example, if the processor 104 wants to prefetch data from the LI cache 106, the system or the stride prefetcher 112 can obtain the average cache miss latency from the LI cache 106 to determine the average latency of the prefetched data.

[0025]

[0023] The duration between cache access requests is the cache access interval, indicating at what frequency the cache access requests are coming. The dynamically computed stride prefetch distance can be computed according to: distance = (average cache miss latency) / (duration between requests). (1)

[0024] In some implementations, the stride prefetch distance can be the next higher integer generated by rounding up the result of the equation (1) to the next integer value.

[0025] In some implementations, the dynamically computed stride prefetch distance can be computed according to: distance = (average cache miss latency) * (access count) / (current access cycle - last access cycle). (2)

[0026] The duration between requests can be a number of cycles between a current access cycle and a last access cycle, divided by an access count. The system can have a single cache access in a single cycle (e.g., access count is one) or multiple cache accesses in a single cycle (e.g., access count is larger than one). In some implementations, when the system has a single cache access in a single cycle, the difference (current access cycle - last access cycle) can be the duration between requests. In some implementations, when the system has multiple cache accesses in a single cycle, the (current access cycle - last access cycle) / (access count) can be the duration between requests. Here, the access count can be used as a factor to increase the distance when there are multiple cache accesses in a single cycle.

[0026]

[0027] The processor 104 and the memory 116 can be integrated onto a single system on a chip (SOC) 102. The processor 104 or the SOC 102 itself can be a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), an ambient computing module, an image processor, a sensor processing module, an application-specific integrated circuit (ASIC), or other lower-level components of the SOC 102 itself that are capable of issuing memory requests to the memory 116. For example, the processor 104 can request data or instructions from a memory or a cache.

[0027]

[0028] The SOC 102 is an example of a device that can be installed on or integrated into any appropriate computing device, which may be referred to as a host device. Because the techniques described in this specification are particularly suited to reducing power consumption and increasing performance for the host device, the SOC 102 can be especially beneficial when installed on mobile host devices that rely on battery power, e.g., a smart phone, a smart watch or another wearable computing device, a tablet computer, or a laptop computer, to name just a few examples.

[0028]

[0029] The system 100 includes a cache subsystem 110. Caches are auxiliary' devices that manage data traffic to memory7. A cache interacts with one or more hardware devices in a system to store data retrieved from memory7, or store data that is to be written to memory, or both. The hardware devices can be various components of an integrated circuit and be implemented into a system on a chip (SOC). Devices that supply read and write requests through caches, or directly to memory7, are referred to as processors 104.

[0029]

[0030] A cache subsystem can be utilized to reduce power consumption by reducing how often requests to main memory are required. Further power savings can be achieved by placing the main memory and the data pathways to main memory in a lowered power state. Due to the inverse correlation between cache usage and power consumption, maximizing cache usage leads to an overall decrease in power consumed. The power capacity of battery powered devices, e.g., mobile computing devices, can be spent more efficiently by increasing cache usage of integrated client devices. Moreover, accessing the cache is generally faster than accessing the main memory, thereby increasing the performance of the integrated client devices.

[0030]

[0031] The cache subsystem 110 includes one or more levels of cache that cache data requests for the processor 104 on the SOC 102. The cache subsystem 110 can include a lower-level cache, e.g., the Level 1 (LI) cache 106, and a higher-level cache, e.g., the Level 2 (L2) cache 108. The lower-level caches can have a smaller number of blocks, smaller block size, fewer blocks in a set, or a combination of these, but have very short access times. The higher-level caches, e.g., Level 2 and above, can have progressively larger number of blocks, larger block size, more blocks in a set, or a combination of these, and relatively longer access times, but are still much faster than the main memory.

[0031]

[0032] For example, the processor 104 can be a CPU and the CPU can have a hierarchy of multiple cache levels, e.g., the LI cache 106 and the L2 cache 108, w ith different instruction-specific and data-specific caches at the LI cache. The LI cache is the one closer to the CPU and the L2 cache is at the next level. The cache memory can be implemented with static random-access memory (SRAM), or other types of memories.

[0033] FIG. 2 is a diagram of an example stride pattern. The average cache miss latency is 10 cycles because the average return time is 10 cycles. The duration between requests is 4 cycles because the requests come every 4 cycles. The access count is 1 access per cycle. Using the formulation in equation (2), the system can calculate 10*1 / 4=2.5, and the stride prefetch distance can be 3, which is the next higher integer generated by rounding up 2.5. If the stride prefetcher wants to do a timely prefetch, the stride prefetcher can start from the stride prefetch distance at 3. For example, if the current cache access is at the first stride, the system can capture the fourth stride to improve timeliness, and can skip the second stride and the third stride.

[0032]

[0034] FIG. 3 is a flowchart of an example process 300 for stride prefetching using a dynamically computed stride prefetch distance. The example process 300 can be performed by one or more components of a system, e.g., the example stride prefetcher 1 12 of the example system 100. The example process 300 will be described as being performed by a stride prefetcher, e.g., the stride prefetcher 112 of the processor 104 on the SOC 102, programmed appropriately in accordance with this specification.

[0033]

[0035] The stride prefetcher determines whether a pattern of memory accesses issued by the processor is a pattern of strided memory accesses (302). In some implementations, the stride prefetcher can determine whether the memory accesses have a fixed address gap. For example, if the stream of memory accesses is A+k. A+2*k, A+3*k, . . . , etc., the stride prefetcher can determine that the pattern of memory accesses is a pattern of strided memory accesses.

[0034]

[0036] FIG. 4A is an example table for a stride prefetcher with a dynamically computed stride prefetch distance. The existing fields can record the “last address’" and “stride” for each program counter (PC). The “stride” is calculated by (current address) - (last address). The stride prefetcher can determine whether the pattern of memory accesses is a strided pattern based on the “last address” field, the “stride” field, or a combination of both. For example, if many PCs have the same “stride” value, the stride prefetcher can determine that the pattern of memory accesses is a pattern of strided memory accesses.

[0035]

[0037] Referring back to FIG. 3, the stride prefetch er dynamically computes a stride prefetch distance based on a measure of cache miss latency and a duration between requests in a sequence of strided memory accesses issued by the processor (304). In some implementations, in response to determining that a pattern of memory accesses issued by the processor is a pattern of strided memory accesses, the stride prefetcher dynamically computes the stride prefetch distance. In some implementations, in response to determining that a pattern of memory accesses issued by the processor is not a pattern of strided memory accesses, the system can process normally 303, e.g., using a regular prefetcher instead of a stride prefetcher.

[0036]

[0038] In some implementations, the duration between requests can be a difference divided by an access count, and the difference can be a number of cycles between a current access cycle and a last access cycle. In some implementations, the dynamically computed stride prefetch distance can be the measure of cache miss latency divided by the duration between access requests. For example, the stride prefetcher can dynamically compute the stride prefetch distance using equation (1) or equation (2).

[0037]

[0039] For example, the system can have a single cache access in a single cycle. There is a 16-byte stride stream of a program counter (PC), which accesses address 0x00 at cycle 0 and address 0x10 at cycle 1. A PC is a register in a computer processor that contains the address of the instruction being executed at the current time. The average cache miss latency is 10. The stride prefetcher can compute the stride prefetch distance using equation (2). For example, the distance can be 10* l / ( 1 -0)=l 0. The stride prefetcher can compute the address of the next data element to be prefetched and the address is 16+16x10=176. That is, the stride prefetcher can prefetch one or more data elements with distance 10, at address OxBO at cycle 1 because the stride prefetcher estimates that at cycle 11, the prefetched line will arrive at the cache and the PC will access the address OxBO.

[0038]

[0040] As another example, the system can have multiple cache accesses in a single cycle. This can happen when there are multiple ports in a cache of the cache subsystem. There is an 8-byte stride stream of a PC, which accesses addresses at 0x00 and 0x08 at cycle 0, addresses 0x10 and 0x18 at cycle 1. The average cache miss latency is 10. The interval access count is 2. The stride prefetcher can compute the stride prefetch distance using equation (2). For example, the distance can be 10*2 / (l-0)=20. The stride prefetcher can compute the addresses of next data elements to be prefetched and the addresses are 16+8*20=176 and 24+8*20=184. That is, the stride prefetcher can prefetch data elements with distance 20, at addresses OxBO and 0xB8 at cycle 1 because the stride prefetcher estimates that at cycle 11, the prefetched line(s) will arrive at the cache and the PC will access the addresses OxBO and 0xB8.

[0039]

[0041] In some implementations, the stride prefetcher can include a stride prefetcher table, and the stride prefetcher table can include a last access cycle field, an access count field, and a stride prefetch distance field. FIG. 4A is an example table for a stride prefetcher with dynamic distance. The table can include several fields for each PC, such as for PCI, PC2, . . . , etc. Some existing fields for a stride pref etcher can include a “last address’’ field and a “stride” field. For a stride prefetcher with dynamic distance, the table can include several additional fields, such as a “last access cycle” field, an “interval access count” field, and a “distance” field.

[0040]

[0042] The "last access cycle” field records the last access cycle. The stride prefetcher can calculate the access interval by (current access cycle) - (last access cycle). The “interval access count” field can be used to handle the situations when there are multiple accesses in a single cycle. For example, if there are two accesses in a cycle, the “interval access count” field is 0.5 cycle. The “distance” field stores the stride prefetch distance obtained after calculation.

[0043] Using these additional fields, the stride prefetcher can fill in the “last access cycle” field and the “interval access count” field for PC. Then, the stride prefetcher can compute the “distance” field using the “last access cycle” field and the “interval access count” field, e.g., following equation (2).

[0041]

[0044] For example, each time the stride prefetcher receives a new program counter (PC), the stride prefetcher can calculate a distance for that new PC. In some implementations, when the “current access cycle” is not equal to “last access cycle”, the stride prefetcher can recalculate the stride prefetch distance.

[0042]

[0045] In some implementations, the measure of the cache miss latency can be average cache miss latency. In some implementations, the stride prefetcher can determine the average cache miss latency by sampling a miss status holding register (MSHR). MSHR is part of the cache subsystem and can belong to a cache level, e.g., LI cache or L2 cache.

[0043]

[0046] FIG. 4B is an example for additional information in a MSHR and cache for dynamic distance adjustment. The MSHR can include multiple fields for each MSHR, such as for MSHR 1, MSHR 2, . . . , etc. Some existing fields for a MSHR can include an “address” field, “targets” field, and some other fields. For a system that includes a stride prefetcher with dynamic distance, the MSHR can include one or more additional fields to record miss latency, such as an “allocate cycle” field. The “allocate cycle” field can record the number of cycles during an allocation and can track the time that the system creates an MSHR entry. The stride prefetcher can store “total latency” and “allocation counts” in the cache. For example, in FIG. 4B, an “AllocateCounf ’ field can track the amount of MSHR entries that have got the data from the lower level memory hierarchies. The stride prefetcher can calculate the average cache miss latency based on the “total latency” field and the “allocation counts” field, e.g., using average cache miss latency = (total latency) / (allocation counts).

[0044]

[0047] For example, when a cache miss occurs, the system can establish an MSHR entry to track the status of the miss data. Meanwhile, the system can set the “allocate cycle” field to the current cycle. When data is returned to the cache, the system can find the corresponding MSHR entry of the returned data. Then, the system can calculate corresponding miss latency following the formula of miss latency = “current cycle of return data - the "allocate cycle’ field from the corresponding MSHR”. Then, the system can add the corresponding miss latency to the “total latency” field. Meanwhile. the system can increase the “allocate count’' field by 1 to count the amount of MSHR entries that have got the data from the lower-level memory hierarchy.

[0045]

[0048] FIG. 5 A is a diagram of a miss status holding register (MSHR). The MSHR can store the MSHR creation time 502 after receiving a demand or a prefetch request. The MSHR can store the MSHR data receive time 504 after receiving data return with fill operation, e.g.. when a corresponding cache line is filled. The stride prefetcher can determine the cache miss latency for this particular MSHR by sampling the MSHR, e g., by calculating (MSHR data receive time) - (MSHR creation time). The stride prefetcher can average this difference in some time window. The techniques in FIG. 5A need two data fields 502 and 504 in MSHR and a subtraction operation. In some implementations, the stride prefetcher can determine the average cache miss latency of a cache level by sampling from partial MSHR instead of all MSHRs. For example, the current cache level can have 100 MSHRs and the stride prefetcher can determine the average cache miss latency of the current cache level by sampling 10 MSHRs out of the 100 MSHRs. Because the miss pattern can spread to all MSHR entries due to the fair allocation policy of MSHR, the system can get approximate cache access latency from part of the MSRH entries, reducing the processing cost of the system. For example, if the observation time frame is long enough, the system can determine the overall average miss latency using part of the MSHR entries.

[0046]

[0049] In some implementations, the stride prefetcher can determine the average cache miss latency based on a data return level and a predefined latency of the data return level. FIG. 5B is a diagram of hit level hint information added in a data return packet. Here, the hit level is the data return level, e.g., cache level N or main memory. The system can add data return level information in the data return packet. The system can determine the corresponding average cache miss latency based on the data return level and a predetermined latency of the data return level. The techniques in FIG. 5B can be a low-cost solution, e.g., adding a few bits overhead per package and no need for a subtraction operation. For example, for a system that has a Ll / L2 / L3 / main memory configuration, adding the data return level information can add approximately 2 bits overhead per packet. The techniques described can eliminate the cost for determining the per entry cycle time by using the hit information from the lower-level memory hierarchy . The miss latency from the different levels can be approximated, e.g., by “bus access latency” + “tag lookup latency” + “data access latency”. Thus, if the system can obtain the hit level of the miss data, the system can calculate the corresponding miss latency without a need to obtain the MSHR creation cycle and the data return time, reducing the processing cost of the system.

[0047]

[0050] For example, the stride prefetcher can have a miss request in LI cache. The stride prefetcher cannot timely prefetch data and store the data in the L 1 cache. Thus, the stride prefetcher cannot satisfy a request from the processor to access the data in the LI cache. The processor can get data from the L2 cache, e.g., a L2 hit. The system can add the data return level information (e.g., cache level L2) to the data return packet. The system can obtain a predefined latency for the L2 cache, e.g., 10 cycles. The system can determine that the cache miss latency for this request is 10 cycles. In some implementations, the system can determine the average cache miss latency over a period of time.

[0048]

[0051] The stride prefetcher prefetches one or more data elements according to the dynamically computed stride prefetch distance (306). The one or more data elements can include data, instructions, or a combination of both.

[0049]

[0052] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly- embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory storage medium for execution by, or to control the operation of, data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. Alternatively or in addition, the program instructions can be encoded on an artificially-generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus.

[0050]

[0053] The term “data processing apparatus” refers to data processing hardware and encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can also be, or further include, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application- specific integrated circuit). The apparatus can optionally include, in addition to hardware, code that creates an execution environment for computer programs, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0051]

[0054] A computer program which may also be referred to or described as a program, software, a software application, an app, a module, a software module, a script, or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub-programs, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a data communication network.

[0052]

[0055] For a system of one or more computers to be configured to perform particular operations or actions means that the system has installed on it software, firmware, hardware, or a combination of them that in operation cause the system to perform the operations or actions. For one or more computer programs to be configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by data processing apparatus, cause the apparatus to perform the operations or actions.

[0053]

[0056] As used in this specification, an "engine." or “software engine,’’ refers to a hardware-implemented or software implemented input / output system that provides an output that is different from the input. An engine can be implemented in dedicated digital circuitry or as computer-readable instructions to be executed by a computing device. Each engine can be implemented within any appropriate type of computing device, e.g., servers, mobile phones, tablet computers, notebook computers, music players, e-book readers, laptop or desktop computers, PDAs, smart phones, or other stationary' or portable devices, that includes one or more processing modules and computer-readable media. Additionally, two or more of the engines may be implemented on the same computing device, or on different computing devices.

[0057] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuitry, e.g., an FPGA or an ASIC, or by a combination of special purpose logic circuitry and one or more programmed computers.

[0054]

[0058] Computers suitable for the execution of a computer program can be based on general or special purpose microprocessors or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read-only memory or a random access memory' or both. The essential elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data. The central processing unit and the memory can be supplemented by, or incorporated in, special purpose logic circuitry. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to. or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.

[0055]

[0059] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory' devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g.. internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0056]

[0060] To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a host device having a displaydevice. e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and pointing device, e.g., a mouse, trackball, or a presence sensitive display or other surface by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory’ feedback, e.g.. visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user’s device in response to requests received from the web browser. Also, a computer can interact with a user by sending text messages or other forms of message to a personal device, e.g., a smartphone, running a messaging application, and receiving responsive messages from the user in return.

[0057]

[0061] In addition to the embodiments described above, the following embodiments are also innovative:

[0058]

[0062] Embodiment 1 is a system comprising: a processor; a stride prefetcher; and a cache subsystem, wherein the stride prefetcher is configured to dynamically compute a stride prefetch distance based on a measure of cache miss latency and a duration between requests in a sequence of strided memory accesses issued by the processor.

[0059]

[0063] Embodiment 2 is the system of embodiment 1 , wherein the stride prefetcher is configured to perform operations comprising: determining that a pattern of memory accesses issued by the processor is a pattern of strided memory accesses; obtaining the dynamically computed stride prefetch distance; and prefetching one or more data elements according to the dynamically computed stride prefetch distance.

[0060]

[0064] Embodiment 3 is the system of embodiment 1, wherein the measure of the cache miss latency is average cache miss latency.

[0061]

[0065] Embodiment 4 is the system of embodiment 3, wherein the stride prefetcher is configured to perform operations comprising: determining the average cache miss latency by sampling a miss status holding register.

[0062]

[0066] Embodiment 5 is the system of embodiment 3, wherein the stride prefetcher is configured to perform operations comprising: determining the average cache miss latency based on a data return level and a predefined latency of the data return level.

[0067] Embodiment 6 is the system of embodiment 1, wherein the duration between requests is a difference divided by an access count, wherein the difference is a number of cycles between a current access cycle and a last access cycle.

[0063]

[0068] Embodiment 7 is the system of embodiment 1, wherein the dynamically computed stride prefetch distance is the measure of cache miss latency divided by the duration between access requests.

[0064]

[0069] Embodiment 8 is the system of embodiment 1 , wherein the stride prefetcher comprises a stride prefetcher table, the stride prefetcher table comprises a last access cycle field, an access count field, and a stride prefetch distance field.

[0065]

[0070] Embodiment 9 is a method, comprising: obtaining a measure of cache miss latency of a cache subsystem; obtaining a duration between requests in a sequence of strided memory accesses issued by a processor; and dynamically computing, by a stride prefetcher, a stride prefetch distance based on the measure of the cache miss latency and the duration between requests in the sequence of strided memory accesses.

[0066]

[0071] Embodiment 10 is one or more non-transitory storage media encoded with instructions that when executed by a stride prefetcher included in a device cause the stride prefetcher to perform operations comprising: obtaining a measure of cache miss latency of a cache subsystem; obtaining a duration between requests in a sequence of strided memory accesses issued by a processor; and dynamically computing a stride prefetch distance based on the measure of the cache miss latency and the duration between requests in the sequence of strided memory accesses.

[0067]

[0072] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or on the scope of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially be claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0068]

[0073] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order show n or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0069]

[0074] Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.

[0075] What is claimed is:

Claims

CLAIMS1. A system comprising: a processor; a stride prefetcher; and a cache subsystem, wherein the stride prefetcher is configured to dynamically compute a stride prefetch distance based on a measure of cache miss latency and a duration between requests in a sequence of strided memory accesses issued by the processor.

2. The system of claim 1, wherein the stride prefetcher is configured to perform operations comprising: determining that a pattern of memory accesses issued by the processor is a pattern of strided memory accesses; obtaining the dynamically computed stride prefetch distance; and prefetching one or more data elements according to the dynamically computed stride prefetch distance.

3. The system of claim 1, wherein the measure of the cache miss latency is average cache miss latency.

4. The system of claim 3, wherein the stride prefetcher is configured to perform operations comprising: determining the average cache miss latency by sampling a miss status holding register.

5. The system of claim 3. wherein the stride pref etcher is configured to perform operations comprising: determining the average cache miss latency based on a data return level and a predefined latency of the data return level.

6. The system of claim 1, wherein the duration between requests is a difference divided by an access count, wherein the difference is a number of cycles between a current access cycle and a last access cycle.

7. The system of claim 1. wherein the dynamically computed stride prefetch distance is the measure of cache miss latency divided by the duration between access requests.

8. The system of claim 1, wherein the stride prefetcher comprises a stride prefetcher table, the stride prefetcher table comprises a last access cycle field, an access count field, and a stride prefetch distance field.

9. A method, comprising: obtaining a measure of cache miss latency of a cache subsystem; obtaining a duration between requests in a sequence of strided memory accesses issued by a processor; and dynamically computing, by a stride prefetcher, a stride prefetch distance based on the measure of the cache miss latency and the duration between requests in the sequence of strided memory accesses.

10. The method of claim 9, comprising: determining that a pattern of memory accesses issued by the processor is a pattern of strided memory accesses; obtaining the dynamically computed stride prefetch distance; and prefetching one or more data elements according to the dynamically computed stride prefetch distance.1 1. The method of claim 9, wherein the measure of the cache miss latency is average cache miss latency.

12. The method of claim 11, comprising: determining the average cache miss latency by sampling a miss status holding register.

13. The method of claim 11, comprising: determining the average cache miss latency based on a data return level and a predefined latency of the data return level.

14. The method of claim 9, wherein the duration between requests is a difference divided by an access count, wherein the difference is a number of cycles between a current access cycle and a last access cycle.

15. The method of claim 9, wherein the dynamically computed stride prefetch distance is the measure of the cache miss latency divided by the duration between access requests.

16. The method of claim 9, wherein the stride prefetcher comprises a stride prefetcher table, the stride prefetcher table comprises a last access cycle field, an access count field, and a stride prefetch distance field.

17. One or more non-transitory storage media encoded with instructions that when executed by a stride prefetcher included in a device cause the stride prefetcher to perform operations comprising: obtaining a measure of cache miss latency of a cache subsystem; obtaining a duration between requests in a sequence of strided memory7accesses issued by a processor; and dynamically computing a stride prefetch distance based on the measure of the cache miss latency and the duration between requests in the sequence of strided memory accesses.

18. The non-transitory storage media of claim 17, wherein the operations comprise: determining that a pattern of memory accesses issued by the processor is a pattern of strided memory accesses; obtaining the dynamically computed stride prefetch distance; and prefetching one or more data elements according to the dynamically computed stride prefetch distance.

19. The non-transitory storage media of claim 17, wherein the measure of the cache miss latency is average cache miss latency.

20. The non-transitory storage media of claim 17, wherein the duration between requests is a difference divided by an access count, wherein the difference is a number of cycles between a current access cycle and a last access cycle.

Images