Prefetching an array of pointers

Abstract

Prefetching an array of pointers is described. According to the described technique, a pointer target instruction is detected by identifying that the load instruction is included in a sequence of load instructions whose addresses are separated by a step size, where the destination location of the load instruction is used in the address calculation of the memory operation. Instructions to fetch the data of the future load instruction are injected into the instruction stream of the processor. The data of the future load instruction is stored in a temporary register. An additional instruction is injected into the instruction stream to prefetch the pointer target based on the address of the memory operation and the data of the future load instruction.

Description

[Technical field]

[0001] (Related Applications) This application claims priority to U.S. patent application Ser. No. 17 / 704,627, entitled "Array of Pointers Prefetching," filed Mar. 25, 2022, the entire disclosure of which is incorporated herein by reference. [Background technology]

[0002] Data prefetching is a technique in which a load-store unit of a processor leverages a prefetch controller to identify data that is likely to be requested in the future by the processor's execution units based on the history of data requested by the execution units. The load-store unit prefetches data from a data store (e.g., main memory) at addresses identified by the prefetch controller and loads the prefetched data into a cache. The prefetched data can be retrieved from the cache faster than the data store.

[0003] The detailed description will now be made with reference to the accompanying drawings. [Brief description of the drawings]

[0004] [Figure 1] FIG. 2 is a block diagram of a non-limiting example of a system having a prefetch controller for prefetching data that is likely to be requested by an execution unit of the system in accordance with one or more embodiments. [Diagram 2] FIG. 2 illustrates a non-limiting example of a representation of an array of pointers. [Figure 3A] FIG. 2 illustrates an example system having a stride prefetcher for prefetching data that is likely to be requested by an execution unit of the system in accordance with one or more embodiments. [Figure 3B] FIG. 2 illustrates in greater detail tables available in an example system in one or more embodiments. [Figure 4] FIG. 2 illustrates a procedure in an exemplary embodiment for injecting instructions into an instruction stream to fetch data for a future load instruction. [Diagram 5] FIG. 2 illustrates a procedure in an exemplary embodiment for injecting instructions into a processor's instruction stream to prefetch data using the address of a memory operation and data from a future load instruction. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0005] (overview) Data prefetching refers to a technique for estimating or predicting data that is likely to be requested in the future by an execution unit of a processor based on the history of data requested by the execution unit. Data predicted to be likely to be requested by an execution unit is prefetched from a data store (e.g., main memory) and loaded into a cache to service requests for data from the cache. When an execution unit requests data that is available in the cache (a cache hit), a load-store unit of the processor can quickly provide the requested data from the cache for processing. However, when an execution unit requests data that is not available in the cache (a cache miss), the requested data is retrieved from a data store, which is slower than serving a request from the cache. Thus, data prefetching is a technique used to load data into a cache before the data is requested to improve the performance of a processor.

[0006] However, one of the main challenges associated with conventional data prefetching techniques is accurately identifying and retrieving the data that will actually be required by an execution unit, so that the data is loaded into the cache and accessible before the execution unit requests the data. This is particularly difficult for arrays of pointers, which are commonly used in applications such as machine learning, graph analytics, and sparse linear algebra. One reason it is difficult to predict what data is likely to be required in the context of arrays of pointers is that the address (e.g., pointer) pointed to by a particular element of the array may depend on data at an address pointed to by a previous element of the array, which may be random and / or may change before the address pointed to by a particular element is requested.

[0007] To overcome these challenges, a decode unit of a processor is modified to selectively inject at least one instruction into an instruction stream to fetch data for a future load instruction. In one or more embodiments, a stride prefetcher of a load store unit of the processor trains the decode unit. For example, the stride prefetcher trains the decode unit by populating a table accessible to the decode unit based at least in part on training events. In one example, the stride prefetcher communicates training events to the table (e.g., over a bus of the processor) in association with a stride prefetch (e.g., triggered based on updates to the stride prefetcher or when it issues a prefetch request).

[0008] In one or more embodiments, the training events include a program counter value, a step size, and a confidence level. As one example, the program counter value is an instruction address and the step size is the difference between consecutive memory addresses accessed by instructions having the same program counter value. In one example, the confidence level is based on the number of times instructions having the same program counter value accessed consecutive memory addresses separated by the step size. As described in more detail below, in one or more embodiments, the training events are used to populate a table based on confidence levels associated with the training events.

[0009] Using the populated table, the decode unit monitors load instructions in the processor's instruction stream and compares the program counter value of the load instruction with the program counter values ​​of entries contained in the table. If the program counter value of the load instruction matches the program counter value of an entry in the table, the destination location (e.g., destination register) of the load instruction is captured and the matching entry in the table is updated to include the destination location.

[0010] The decode unit monitors the address calculations of input instructions in the decoded instruction stream to identify use of the destination location by the input instructions. In various scenarios, the destination location is used directly, such as for an index or base register in the input instruction's address calculation. Alternatively, the destination location is used indirectly, such as when the destination location is used in a separate instruction and the result of the separate instruction is used in the input instruction's address calculation.

[0011] A pointer target instruction is identified when the decode unit identifies a use of the destination location of the load instruction by an incoming instruction in the instruction stream. Upon identifying the pointer target instruction, the pointer array prefetcher inserts an instruction to fetch data of a future load instruction in the decode instruction stream via an inject bus. In one or more embodiments, the inserted instruction includes an address determined by offsetting the address of the load instruction by the product of the step size and the step number (from the matching entry in the table).

[0012] The load store unit receives the inserted instruction and fetches the data of the future load instruction using the address included in the inserted instruction. For example, the load store unit writes the data of the future load instruction to a temporary location (or register) of the processor available to the pointer array prefetcher and / or the decode unit. The pointer array prefetcher then injects an additional instruction into the instruction stream to prefetch the future pointer target. In one or more embodiments, this additional instruction includes an address that is the address of the input instruction (e.g., the pointer target instruction) with the temporary register (e.g., the data of the future load instruction) used in place of the destination location for the address calculation.

[0013] The load store unit receives the additional instructions and uses their addresses to prefetch pointer targets. According to the described techniques, the prefetched pointer targets are loaded into the cache before an execution unit requests a future pointer target. Thus, when an execution unit requests a pointer target, the load store unit can quickly provide the pointer target from the cache. By prefetching pointer targets in this manner, the described techniques generate more cache hits and therefore improve processor performance over conventional techniques for prefetching data.

[0014] In some aspects, the techniques described herein relate to a method that includes detecting a pointer target instruction by identifying that a destination location of a load instruction included in a sequence of load instructions having addresses separated by a step size is used in an address calculation for a memory operation, and injecting an instruction into the instruction stream of a processor to fetch data for a future load instruction using the address of the load instruction offset by a distance based on the step size.

[0015] In some aspects, the techniques described herein relate to a method, where a pointer target instruction is detected in a decode unit of an instruction stream.

[0016] In some aspects, the techniques described herein relate to a method that further includes storing data for a future load instruction in a temporary register or location accessible to a decode unit of the processor.

[0017] In some aspects, the techniques described herein relate to a method, where the memory operation is a load, a load and execute, a store, or a floating point load.

[0018] In some aspects, the techniques described herein relate to a method, where the distance is a product of the step size and the number of steps.

[0019] In some aspects, the techniques described herein relate to a method whereby instructions are injected into an instruction stream via an inject bus of a processor.

[0020] In some aspects, the techniques described herein relate to a method that further includes injecting additional instructions into the instruction stream to prefetch pointer targets based on the address of the memory operation and data of future load instructions.

[0021] In some aspects, the techniques described herein relate to a method further including storing the pointer target in a cache.

[0022] In some aspects, the techniques described herein relate to a method further including tracking miss rates of the L1 cache.

[0023] In some aspects, the techniques described herein relate to how additional pointer target instructions are detected based on a miss rate of an L1 cache.

[0024] In some aspects, the technology described herein relates to how additional pointer target instructions are not detected based on the miss rate of the L1 cache.

[0025] In some aspects, the technology described herein relates to a system that includes a decode unit of a processor configured to identify that a destination location of a load instruction included in a sequence of load instructions having addresses separated by a step size is to be used in an address calculation of a memory operation, and a pointer array prefetcher of the processor configured to inject instructions into the instruction stream of the processor to prefetch data using the address of the memory operation and data of a future load instruction fetched based on the step size.

[0026] In some aspects, the techniques described herein relate to a system, where the memory operation is a load, a load and execute, a store, or a floating point load.

[0027] In some aspects, the techniques described herein relate to a system where data is stored in an L1 cache, an L2 cache, or an L3 cache.

[0028] In some aspects, the techniques described herein relate to a system where data for a future load instruction is fetched using the address of the load instruction offset by a distance based on a step size.

[0029] In some aspects, the techniques described herein relate to a system where the distance is the product of the step size and the number of steps.

[0030] In some aspects, the techniques described herein relate to a system where data for future load instructions is stored in a temporary register or location accessible to a decode unit.

[0031] In some aspects, the techniques described herein relate to a method that includes identifying a destination location of a load instruction in a sequence of load instructions having addresses separated by a step size to be used in an address calculation of a memory operation, and injecting an instruction into the instruction stream of a processor to prefetch data using the address of the memory operation and data of a future load instruction that is fetched based on the step size.

[0032] In some aspects, the techniques described herein relate to a method, where data for a future load instruction is fetched using the address of the load instruction offset by a distance based on a step size.

[0033] In some aspects, the techniques described herein relate to a method, where the distance is a product of the step size and the number of steps.

[0034] 1 is a block diagram of a non-limiting exemplary system 100 having a prefetch controller for prefetching data likely to be required by execution units of the system in accordance with one or more embodiments. In particular, system 100 includes a fetch unit 102, a decode unit 104, an execution unit 106, and a load store unit 108 of a processor.

[0035] In one or more embodiments, a program counter (not shown) of the processor indicates instructions to be processed by the processor as part of instruction stream 110. By way of example, fetch unit 102 fetches instructions indicated by the program counter, and decode unit 104 decodes the fetched instructions for execution by execution unit 106. In at least one variation, the program counter is incremented after an instruction is fetched to indicate the next instruction to be executed as part of instruction stream 110.

[0036] According to the described technique, the execution unit 106 requests data to execute an instruction. In a variant, the cache 112 is first searched for the requested data. In one or more embodiments, the cache 112 is a memory cache, such as a particular level of cache (e.g., an L1 cache or an L2 cache), which is included in a hierarchy of multiple cache levels (e.g., L0, L1, L2, L3, L4). If the requested data is available in the cache 112 (e.g., a cache hit), the load store unit 108 can quickly provide the requested data from the cache 112. However, if the requested data is not available in the cache 112 (e.g., a cache miss), the requested data is retrieved from a data store, such as the memory 114.

[0037] It should be appreciated that memory 114 (e.g., random access memory) is one example of a data store from which data may be retrieved if not already stored in cache 112 and / or from which data may be loaded into cache 112, e.g., using the prefetching techniques described above and below. Other examples of data stores include, but are not limited to, external memory, higher level caches (e.g., an L2 cache if cache 112 is an L1 cache), secondary storage (e.g., mass storage devices), and removable media (e.g., flash drives, memory cards, compact disks, digital video disks), to name a few. In particular, serving requested data from a data store when a cache miss occurs is slower than serving requested data from cache 112 when a cache hit occurs.

[0038] To avoid cache misses that increase latency, the load store unit 108 includes a prefetch controller 116 that identifies patterns in memory addresses that are accessed when the execution units 106 execute instructions. The identified patterns can be used to determine memory addresses in the memory 114 that contain data that the execution units 106 are likely to request in the future. The prefetch controller 116 and / or the load store unit 108 "prefetch" data from the determined memory addresses in the memory 114 and store the prefetched data in the cache 112, for example, before the execution units 106 request the prefetched data for execution of an instruction of the instruction stream 110 that uses the prefetched data. In accordance with the described techniques, for example, the data that is requested and prefetched in connection with executing an instruction corresponds to an array of pointers (e.g., at least one pointer target), an example of which is described in more detail with respect to FIG. 2.

[0039] Prefetch controller 116 can identify various different types of patterns in memory addresses accessed when execution unit 106 executes instructions included in instruction stream 110. In the illustrated example, prefetch controller 116 includes various prefetchers that correspond to examples of those different types of patterns. However, it should be understood that in one or more embodiments, prefetch controller 116 includes fewer, more, or different prefetchers without departing from the spirit or scope of the described technology. By way of example and not limitation, prefetch controller 116 includes next line prefetcher 118, stream prefetcher 120, stride prefetcher 122, and other prefetcher 124.

[0040] In one or more embodiments, next line prefetcher 118 identifies a request for a line of data and prefetches the next line of data for loading into cache 112 (e.g., by communicating a prefetch instruction to load store unit 108). Stream prefetcher 120 can prefetch data multiple lines ahead of the requested data by identifying a first data access of a stream, determining a direction of the stream based on a second data access of the stream, and then determining based on a third data access that the first, second, and third data accesses are associated with the stream, etc. Based on this, stream prefetcher 120 initiates prefetching of data for the stream, e.g., by communicating at least one prefetch instruction to load store unit 108.

[0041] Stride prefetcher 122 is similar to stream prefetcher 120, but stride prefetcher 122 is capable of identifying memory address access patterns that follow a "stride" or "step size," such as by identifying a pattern in some memory location between the beginning of the locations where data is accessed. In one or more embodiments, the "stride" or "step size" is measured in bytes or other units.

[0042] In one example, the stride prefetcher 122 identifies a location in memory (e.g., a first memory address) of a beginning of a first element associated with the access. In this example, the stride prefetcher 122 determines a direction and a "step size" or "stride" based on a location in memory of a beginning of a second element associated with the access (e.g., a second memory address), whereby the stride or step size corresponds to a number of locations in memory between the beginning of the first element and the beginning of the second element. In one or more embodiments, the stride prefetcher 122 confirms the pattern based on further determining that a location in memory of a beginning of a third element associated with the access (e.g., a third memory address) is also a "stride" or "step size" away from the location in memory of the beginning of the second element. The stride prefetcher 122 is then configured to initiate prefetching of the respective data based on the stride or step size.

[0043] In at least one variant, the stride prefetcher 122 stores a program counter value, a stride or step size, and / or other information, examples of which include a confidence level and a virtual address. In the illustrated example, the load store unit 108 is shown to include or otherwise have access to a table 126. Further, the table 126 is shown to have entries having a valid field 128, a program counter field 130, a stride field 132, and other fields 134. In one or more embodiments, the table 126 includes one or more entries corresponding to at least one sequence of instructions processed by the system 100. The inclusion of ellipses in the figure represents the ability of the table 126, in at least one variant, to maintain more than one entry. For each entry in the table 126 associated with an instruction sequence, a respective value is stored in a field of the table 126, such as the valid field 128, the program counter field 130, the stride field 132, and / or other fields 134.

[0044] In one example, an entry in table 126 corresponds to a sequence of load and store instructions. In program counter field 130, load store unit 108 or prefetch controller 116 stores a program counter value that, in one or more scenarios, is an instruction address shared by instructions in the sequence of instructions (e.g., sequential instructions). In at least one variation, only a portion of the program counter value is stored in the entry's program counter field 130, e.g., to reduce the number of bits used to store the entry in table 126 compared to including the entire program counter value in the entry's field. In another example, a program counter hash value is calculated from the program counter value (e.g., using a hash function) and stored in program counter field 130 to reduce the number of bits used to store the entry in table 126.

[0045] In the stride field 132, the load store unit 108 or prefetch controller 116 stores a determined step size between locations in memory (e.g., memory addresses) accessed at the beginning of elements of the array for instructions in a sequence of instructions (e.g., sequential instructions). In one or more embodiments, the table 126 stores other information about the entries in other field 134, such as a confidence level, a virtual address, and various other information. As an example, the other information includes the number of memory addresses accessed by an instruction in a sequence of instructions separated by the step size indicated in the stride field 132.

[0046] Other prefetcher 124 represents additional data prefetching functionality. In one or more variations, for example, other prefetcher 124 is capable of correlated prefetching, tag-based correlated prefetching, and / or pre-execution based prefetching, to name a few. In one or more embodiments, the prefetching functionality of other prefetcher 124 is used to augment or replace the functionality of next line prefetcher 118, stream prefetcher 120, and / or stride prefetcher 122. In the context of prefetching an array of pointers, consider the following example.

[0047] 2 illustrates a non-limiting example 200 of a representation of an array of pointers. In this example 200, the representation shows a first memory address 202, a second memory address 204, a third memory address 206, a fourth memory address 208, and a fifth memory address 210, which correspond to locations in memory of the beginning of elements of an array 212. Furthermore, the elements of array 212 are pointers to locations in memory (e.g., memory addresses) that contain data values.

[0048] In the example 200, the elements of the array 212 include a first pointer 214, a second pointer 216, a third pointer 218, a fourth pointer 220, and a fifth pointer 222. As shown, a first memory address 202 corresponds to the beginning of a first element of the array 212, which in this example is the first pointer 214. The first pointer 214 further points to an address "1000" in memory that contains data 224 (value "10") in this example. Also in this example, a second memory address 204 corresponds to the beginning of a second element of the array 212, which in this example is the second pointer 216, which points to an address "1008" in memory that contains data 226 (value "250"). The third memory address 206 corresponds to the beginning of the third element of the array 212, the third element of the array being a third pointer 218, which in this example points to an address "1070" in memory containing data 228 (value "40"). The fourth memory address 208 corresponds to the beginning of the fourth element of the array 212, the fourth element of the array being a fourth pointer 220, which in this example points to an address "1050" in memory containing data 230 (value "10"). The fifth memory address 210 corresponds to the beginning of the fifth element of the array 212, the fifth element of the array 212 being a fifth pointer 222, which in this example points to an address "1030" in memory containing data 232 (value "300"). It should be understood that array 212 is merely an example, and that the techniques described may operate on arrays of different sizes and refer to different types of data, e.g., strings, floating point values, and various other data structures, without departing from the spirit or scope of the techniques described herein.

[0049] In this example 200, the difference between memory addresses 202-210 corresponding to the locations in memory of the beginning of successive elements of array 212 is 4 (e.g., 4 bytes). Thus, in this example 200, the stride or step size of array 212 is 4. Thus, memory addresses 202-210 are predictable using a difference of 4. If array 212 includes a sixth element (not shown), then the sixth memory address at which the sixth element (e.g., a sixth pointer) of array 212 begins is likely to be equal to fifth memory address 210 (e.g., "116") + 4 (e.g., or "120"). It should be understood that in various systems, depending on various conditions, the difference in memory addresses corresponding to the locations in memory of the beginning of successive elements of the array (e.g., of a pointer) may be different than 4 without departing from the spirit or scope of the described technology.

[0050] Unlike memory addresses 202-210, which are predictable using a difference of four, data 224-232 do not follow such a pattern in the illustrated example because, for example, the values ​​are random, are not tied to a pattern relative to one another, are subject to dynamic behavior, etc. As discussed above, for example, data 224 corresponds to a value "10," data 226 corresponds to a value "250," data 228 corresponds to a value "40," data 230 corresponds to a value "10," and data 232 corresponds to a value "300." In the context of prefetching an array of pointers, consider the following example.

[0051] 3A and 3B show a non-limiting example 300 of a system for prefetching an array of pointers.

[0052] 3A illustrates an example system 300 having a stride prefetcher for prefetching data likely to be required by an execution unit of the system, according to one or more embodiments. In particular, the example system 300 includes a fetch unit 102, a decode unit 104, an execution unit 106, a load store unit 108, a cache 112, and a stride prefetcher 122. In one or more embodiments, the example 300 system includes a pointer array prefetcher 302. In this example 300, the pointer array prefetcher 302 includes or otherwise has access to a table 304 and a table 306. Alternatively or additionally, the decode unit 104 includes or otherwise has access to the tables 304 and 306 shown in more detail in FIG. 3B.

[0053] FIG. 3B illustrates in more detail the tables available to the example system 300 in one or more embodiments. In particular, FIG. 3B illustrates in more detail the tables 304 and 306. In one or more embodiments, the stride prefetcher 122 populates the table 304. In this example, the table 304 includes entries having a valid field 308, a program counter field 310, a stride field 312, an active field 314, a destination register field 316, a trained field 318, and a stride load register number field 320. It should be understood that in one or more embodiments, the table 304 includes different fields without departing from the spirit or scope of the described technology. The table 304 is illustrated with an oval to indicate that the table 304 can maintain multiple entries having such fields.

[0054] In this example, table 306 includes a valid field 322, a program counter field 324, a used index register field 328, a confidence field 330, an other source register field 332, an other source register confidence field 334, a stride load entry field 336, a cache status field 338, and a cache miss field 340. It should be understood that in one or more embodiments, table 306 includes different fields without departing from the spirit or scope of the described technology. Table 306 is shown with an oval to indicate that table 306 may maintain multiple entries having such fields.

[0055] In the context of populating table 304, stride prefetcher 122 communicates (e.g., over the processor's bus) a training event to table 304 each time stride prefetcher 122 makes a prefetch request, the training event including a program counter value, a step size (stride), and a confidence level. The program counter value is an instruction address, and the step size is the difference between consecutive memory addresses accessed by instructions (e.g., instructions in a loop) with the same program counter value. The confidence level is the number of times that instructions with the same program counter value access consecutive memory addresses separated by the step size.

[0056] To populate table 304, the program counter value of each training event is compared to the program counter value stored in the program counter field 310 of each entry in table 304. The program counter value of the training event either matches the program counter value stored in the program counter field 310 of at least one entry in table 304 or does not match the program counter value stored in the program counter field 310 of any of the entries in table 304.

[0057] According to the described technique, the stride prefetcher 122 populates the table based in part on the confidence level of the training event. In one example, a training event matches an entry in the table 304 (e.g., if the program counter value of the training event matches the program counter value in the program counter field 310 of the entry). If the confidence level of the training event is low (e.g., does not meet a threshold confidence level), the entry is invalidated by setting the value stored in the valid field 308 to indicate that the entry is invalid. In one or more embodiments, the valid field 308 corresponds to a validity bit, and the entry is invalidated by setting the validity bit of the validity field 308 equal to "0". In contrast, in one or more embodiments, if the validity bit of the validity field 308 is equal to "1", the entry is valid. It should be understood that the validity field 308 may indicate validity and invalidity in other ways without departing from the spirit or scope of the described technique. In a scenario where a training event matches an entry in the table and the confidence level of the training event is high (e.g., meets a threshold confidence level), the step size of the training event can be used to update the stride field 312 of the respective entry, for example, if the step size of the training event does not match the step size already stored in the stride field 312 of the entry.

[0058] In one example, a training event does not match an entry in table 304 if, for example, the program counter value of the training event does not match the program counter value in any entry's program counter field 310. In this example, if the confidence level of the training event is low (e.g., does not meet a threshold confidence level), the training event is discarded and table 304 is not updated based on the training event. Instead, the program counter value of the subsequent training event is compared to the program counter value contained in the program counter field 310 of the entry in table 304.

[0059] In contrast to the scenario described above, if the confidence level of the non-matching training event is high (e.g., meets a threshold confidence level), a new entry is added to table 304 and the valid field 308 is set to indicate that the new entry is valid, e.g., by setting the validity bit of the valid field 308 of the new entry equal to "1". The new entry in table 304 is further populated based on the training event. For example, the program counter field 310 of the new entry in table 304 is updated to store the program counter value of the training event, and the stride field 312 of the new entry in table 304 is updated to store the step size of the training event.

[0060] After table 304 is populated based on training events from stride prefetcher 122, decode unit 104 further updates table 304. According to the described technique, decode unit 104 updates table 304 by comparing the program counter values ​​of load instructions in instruction stream 110 with the program counter values ​​contained in program counter fields 310 of entries in table 304, e.g., by using content addressable memory such that the comparison is completed quickly, e.g., in one clock cycle. In one or more embodiments, the load instruction whose value is compared is a “younger” instruction, which in at least one example is an instruction that is received after table 304 has been populated with an entry having a matching program counter value.

[0061] In this context, instruction stream 110 is shown to include a load instruction 342. In accordance with the described techniques, the load instruction is processed to read data from a data store (e.g., memory) and load (e.g., copy) the data to a destination location (e.g., a destination register) so that the data can be used by execution unit 106. In the illustrated example 300, the load instruction 342 includes a program counter value 344, a destination location 346, and a data source 348, e.g., from which the data is to be loaded into the specified destination location 346. One example of the data source 348 is a memory address.

[0062] If the program counter value 344 of the load instruction 342 matches a value in the program counter field 310 of an entry in the table 304, the decode unit 104 activates the respective entry. For example, the decode unit 104 sets a value in the active field 314 of the matching entry to indicate that the entry is active. In one or more embodiments, the active field 314 is configured as a bit, such that setting or leaving the bit at a value of "1" indicates that the entry is active and setting or leaving the bit at a value of "0" indicates that the entry is inactive. In variations, the active field 314 is configured to indicate activation and deactivation in other ways.

[0063] In addition to activating the matched entry, decode unit 104 updates destination register field 316 of the matched entry to include a destination location 346 (e.g., a destination register) of load instruction 342. If a subsequent instruction in instruction stream 110 is processed that has a program counter value 344 that matches an entry in table 304 but has a different destination location 346 than the matched entry, decode unit 104 causes the destination register field 316 of the matched entry to be overwritten with the different destination location. Decode unit 104 also deactivates the entry, for example, by updating active field 314 to indicate that the entry is inactive.

[0064] While an entry in table 304 is activated, decode unit 104 monitors address calculations of incoming load and store instructions (e.g., load instructions, store instructions, load and execute instructions, floating point load instructions, etc.) in instruction stream 110 and determines whether the monitored address calculation uses destination location 346. As used herein, the term "address calculation" refers to a specification of or a reference to an equation used to calculate an address, rather than a specification of an actual address that in one or more scenarios is not available at decode time.

[0065] For example, decode unit 104 monitors the address calculations of incoming instructions in instruction stream 110 for any use (either direct or indirect) of destination location 346 specified in the load instruction and maintained in an entry in table 304. As described above and below, destination location 346 may in one example be directly available for use, such as an index or base register, for the monitored address calculation associated with the incoming instruction. In another example, destination location 346 of load instruction 342 may be indirectly available in a separate instruction, such as a load effective address, an add or subtract instruction, and the monitored address calculation of the incoming instruction directly uses the result of the separate instruction.

[0066] In one or more embodiments, instruction stream 110 includes subsequent instructions 350 that include respective program counter values ​​352, respective destination locations 354 (e.g., destination registers), and, in one or more scenarios, respective data sources that correspond to address calculations 356. As an example, address calculation 356 of subsequent instructions 350 uses destination location 346 of load instruction 342. For example, in this example, load instruction 342 may be “mov P1,[B1+I1 * s1+D1]' and the subsequent instruction 350 is 'mov P2,[B2+P1 * Consider the destination location 346 of the load instruction 342 to be represented as “P1”, and the address calculation 356 of the successor instruction 350 to be “[B2+P1 * s2+D2]". It is observed that the destination location 346, i.e., "P1", of the load instruction 342 is used in the address calculation 356 of the subsequent instruction 350. In part, the subsequent instruction 350 is a pointer target instruction because the destination location 346 of the load instruction 342 is used in the address calculation 356 of the subsequent instruction 350. In particular, the destination location 346 is used as an index register in the address calculation 356 in this example. However, it should be understood that the destination location 346 may be used as a base register for the address calculation of the instruction in other examples. In one or more embodiments, the decode unit 104 detects that the subsequent instruction 350 is a pointer target instruction based on the use of the destination location 346 of the load instruction 342 in the address calculation 356.

[0067] Based on decode unit 104 detecting that subsequent instruction 350 is a pointer target instruction, decode unit 104 causes an entry for the instruction to be added to table 306. As an example, decode unit 104 causes program counter value 352 to be entered into program counter field 324 of the entry added to table 306 and causes valid field 322 to be updated to indicate that the entry is valid (e.g., sets the validity bit equal to "1"). In relation to the above example in which destination location 346 of load instruction 342 is used as an index register in address calculation 356, used index register field 328 is updated to indicate that destination location 346 is used as an index register, e.g., the bit corresponding to the field is set to "true" or "1". In an example in which destination location 346 is used as a base register instead of an index register, used index register field 328 is updated to indicate that destination location 346 is used as a base register or is not used as an index register. In scenarios where the used index register field 328 corresponds to a bit, that bit can be updated to "false" or "0" to indicate that the destination location 346 in the pointer target instruction is used as a base register rather than as an index register.

[0068] Based on adding an entry to the table 306, the load instruction 342 is "paired" with a subsequent instruction 350 that corresponds to the pointer target instruction. In accordance with the described technique, an identifier (e.g., index) of the entry in the table 304 that corresponds to the load instruction 342 is used to populate the stride load entry field 336 of the entry for the subsequent instruction 350 in the table 306. In operation, the load instruction 342 may be paired with multiple instructions identified as pointer target instructions because the load instruction 342 uses the destination location 346 of the load instruction 342 in their respective address calculations. It should be understood that in some examples, a particular instruction identified as a pointer target instruction may be "paired" with multiple other instructions, such as the load instruction 342 and additional load instructions. Although the instruction stream 110 is shown with only a single load instruction and a single subsequent instruction, it should be understood that the instruction stream 110 includes multiple load instructions and other types of instructions (e.g., pointer target instructions) in operation.

[0069] In one example, the confidence field 330 maintains a confidence count for each entry in the table 306 such that a confidence count corresponding to each entry is maintained by its respective confidence field 330. It should be understood that in one or more embodiments, confidence is measured and indicated in other ways from the count. However, in at least one embodiment in which confidence counts are used, the confidence count of the entry corresponding to the successor instruction 350 is incremented each time an instance of the successor instruction 350 identified as a pointer target instruction is detected between instances of the load instruction 342 and is decremented if there is not a detection. In a scenario in which an instruction identified as a pointer target instruction corresponds to an entry in the table 306 and has an additional source register, the other source register field 332 of the entry is updated to indicate the additional source register and the other source register confidence field 334 is maintained for the entry. For example, the additional confidence count for the entry corresponding to the successor instruction 350 is decremented each time the corresponding source register is overwritten between two of its instances and is incremented otherwise.

[0070] In one or more embodiments, the trained field 318 of an entry in table 306 is based on at least one of the confidence field 330 and the other source register confidence field 334 of the corresponding entry in table 304. For example, the confidence 330 and / or the other source register confidence 334 are compared to a threshold confidence. If the confidence 330 and / or the other source register confidence 334 of the corresponding entry in table 306 meets the confidence threshold, the trained field 318 of the entry is updated to indicate that the entry has been trained.

[0071] In the context of paired load instruction 342 and successor instruction 350, for example, if the confidence field 330 and / or other source register confidence field 334 of the entry for successor instruction 350 in table 304 meet their respective confidence thresholds, then trained field 318 of the entry for load instruction 342 in table 306 is set to indicate that load instruction 342 is “trained.” For example, the bit corresponding to trained field 318 is set to a value of “1.” As a result of being “trained,” load instruction 342 becomes eligible to be inserted into instruction stream 110 to fetch data for a future load instruction.

[0072] Based on this, the pointer array prefetcher 302 and / or the decode unit 104 use the temporary register or stride load register number for data stores associated with fetching data for a future load instruction corresponding to the load instruction 342, for example. Additionally, the stride load register number field 320 of the entry for the load instruction 342 is updated with the temporary register or stride load register number. If the temporary register is not used, it is returned to the integer register pool. This is useful for applications that do not include a stride prefetcher, since the temporary register is not reserved if it is not used.

[0073] In one or more embodiments, the pointer array prefetcher 302 inserts an instruction to fetch data of a future load instruction via the inject bus 358. If the load instruction 342 is eligible for instruction insertion, if the trained field 318 of its corresponding entry in the table 306 indicates that the instruction has been trained, and if the confidence field 330 of the entry in the table 304 associated with the load instruction 342 (e.g., via the stride load entry field 336) meets a confidence threshold, the load instruction 342 is in “insertion mode.” While the load instruction 342 is in “insertion mode,” the pointer array prefetcher 302 inserts an address load instruction 360 to fetch data of the future load, e.g., data 224-232 pointed to by pointers 214-222 at memory addresses 202-210 in the example 200. In particular, the pointer array prefetcher 302 inserts the address load instruction 360 into the instruction stream 110 via the inject bus 358.

[0074] In one or more embodiments, the address load instruction 360 includes a temporary register number maintained in the stride load register number field 320 and an address in a data store (e.g., memory 114) that is determined based on the step size indicated in the data source 348 and stride field 312 of the load instruction 342. For example, the address of the address load instruction 360 is calculated by adding the address indicated in the data source 348 of the load instruction 342 to the product of the step size and the step number indicated in the stride field 312. The step number is variable in different embodiments. In one or more embodiments, for example, the step number is a constant. Alternatively, the step number is adjustable, for example starting at a first number and increasing to at least a second, larger number. Alternatively or additionally, the step number is constrained in one or more ways. For example, the number of steps may be determined as a power of two (e.g., such that the number is generated using shift operations rather than multiplication operations), the number of steps may be based on the type of cache 112 (e.g., the number of steps is smaller for a lower level cache (L1) than for a higher level cache (L2+), etc. In one or more embodiments, the number of steps may be adjustable, for example, in the code of the program in which instruction stream 110 is being processed.

[0075] In response to receiving the address load instruction 360, the load store unit 108 fetches data for the future load instruction from the data store address indicated in the address load instruction 360 and loads the fetched data into at least a portion of the temporary register indicated in the address load instruction 360. The temporary register is available to the decode unit 104 and / or the pointer array prefetcher 302. In one or more embodiments, the load store unit 108 does not train the stride prefetcher 122 based on the inserted address load instruction 360. In a variation, the load store unit 108 trains the stride prefetcher 122 based on the inserted address load instruction 360.

[0076] In accordance with the described techniques, the pointer array prefetcher 302 inserts additional instructions into the instruction stream 110 via the inject bus 358. Specifically, the pointer array prefetcher 302 inserts a pointer target prefetch instruction 362 to prefetch the target of at least one pointer in the array being prefetched, e.g., an item of data 224-232 pointed to in example 200. In one or more embodiments, the pointer array prefetcher 302 issues the pointer target prefetch instruction 362 based on (1) detecting a successor instruction 350 corresponding to the pointer target instruction, (2) the pointer target entry being “trained,” and (3) the corresponding entry in table 304 being in insert mode.

[0077] Depending on whether the destination location 346 of the load instruction 342 is used as a base register or an index register in the subsequent instruction 350, the format of the pointer target instruction is affected. For example, if the destination location 346 of the load instruction 342 is used as an index register in the subsequent instruction 350, the temporary register indicated in the stride load register number field 320 replaces the destination location 346 as an index register in the pointer target prefetch instruction 362. In contrast, if the destination location 346 of the load instruction is used as a base register in the subsequent instruction 350, the temporary register indicated in the stride load register number field 320 replaces the destination location 346 as a base register in the pointer target prefetch instruction 362. After receiving the pointer target prefetch instruction 362, the load store unit 108 prefetches future pointer targets, e.g., items of data 224-232 pointed to in the example 200. As an example, the address load instruction 360 causes data 228 to be fetched, and the pointer target prefetch instruction 362 causes data 232 to be prefetched. In essence, the address load instruction 360 causes the load store unit 108 to load data from an array (e.g., pointers 214-222) into a temporary register, and the pointer target prefetch instruction 362 causes the load store unit 108 to prefetch future pointer targets (e.g., items of data 224-232) for loading into the cache 112 (e.g., the L1 cache).

[0078] In one example, the load store unit 108 writes the prefetched pointer target to the cache 112 before the execution unit 106 requests the future pointer target. In this way, when the execution unit 106 requests the future pointer target, the load store unit 108 can quickly provide the pointer target by accessing the cache 112. This avoids the latency that would occur if the pointer target was not available in the cache 112 and instead must be loaded, for example, from a data store, such as memory 114. By leveraging the tables 304, 306 and the pointer array prefetcher 302 in this manner, the described system increases the amount of time that data associated with the array of pointers is maintained in the cache 112. Thus, when processing an instruction stream that includes an array of pointers, the described system generates more cache hits and therefore improves processor performance over conventional techniques for prefetching data for the array of pointers.

[0079] In one or more embodiments, because inserting instructions into instruction stream 110 via inject bus 358 consumes processor resources, pointer array prefetcher 302 does not continuously insert instructions into inject bus 358 under all conditions. In at least one variant, a pair of counters is maintained for each entry in table 306, e.g., using cache status field 338 and cache miss field 340. The counters are updated based on hit / miss status received from load store unit 108 for a cache, such as cache 112, which corresponds to the L1 data cache in one or more variants. For example, this tracks the miss rate of the L1 data cache. For example, the cache status counter maintained using cache status field 338 is incremented each time a status is received from load store unit 108. Additionally, the cache miss counter maintained using cache miss field 340 is incremented each time a status is received that is a cache miss.

[0080] In one or more embodiments in which a cache status 338 and a cache miss field 340 are included in an entry in table 306, the cache status counter is monitored against a threshold. If the cache status counter meets the threshold, one or more of the cache status counter and the cache miss counter are divided by two by shifting right. Additionally or alternatively, an entry in table 306 is turned off or trained based on the counters, for example, if its cache status counter is greater than a status count threshold and the miss rate is less than a miss rate threshold. Alternatively or in addition, the two counters are reset periodically after a threshold number of cycles, such as every 100,000 cycles.

[0081] By using decode unit 104 to monitor address calculations of incoming load and store instructions in instruction stream 110, and by inserting prefetch instructions for arrays of pointers (e.g., address load instruction 360 and pointer target prefetch instruction 362), the described system compares register numbers of the inserted instructions rather than memory addresses. Because register numbers are typically smaller than memory addresses (e.g., 5 bits vs. 64 bits), the hardware required to prefetch arrays of pointers by the described system is reduced relative to conventional techniques for prefetching arrays of pointers. Evaluation of the described techniques demonstrates substantial improvements in processor performance relative to a baseline that does not incorporate the described techniques. Furthermore, the described techniques are implementable using a variety of different types of processors and using different cache levels of a particular processor.

[0082] FIG. 4 illustrates a procedure 400 in an exemplary embodiment for injecting instructions into an instruction stream to fetch data for a future load instruction.

[0083] A pointer target instruction is detected (block 402). According to principles described herein, the pointer target instruction is detected by identifying that the destination location of the load instruction is used in an address calculation for the memory operation. For example, the address calculation is a function or formula that determines an address for the memory operation that is not yet available in the decoded instruction stream. The memory operation may be a load, load and execute, store, floating point load, etc., e.g., the load instruction is included in a sequence of load instructions that have the same program counter value and access consecutive memory addresses separated by a step size. In one example, the destination location of the load instruction is used directly in the address calculation for the memory operation, such as a base register or index register for the address of the memory operation. In another example, the destination location of the load instruction is used indirectly in the address calculation for the memory operation, such as in an example where at least a portion of the address calculation is performed using a separate instruction and a result of the separate instruction is included in the address calculation for the memory operation. For example, the separate instruction may be an effective address load instruction, an add instruction, a subtract instruction, etc.

[0084] An instruction is injected into the processor's instruction stream to fetch data for the future load instruction using the address of the load instruction offset by a distance based on the step size (block 404). For example, pointer array prefetcher 302 injects an instruction (e.g., address load instruction 360) via inject bus 358. In one example, load store unit 108 receives the instruction and fetches the data for the future load instruction, and load store unit 108 writes the data for the future load instruction to a temporary register of the processor.

[0085] FIG. 5 illustrates a procedure 500 in an exemplary embodiment for injecting instructions into a processor's instruction stream to prefetch data using the address of a memory operation and the data of a future load instruction.

[0086] The use of the destination location of the load instruction in the address calculation of the memory operation is identified (block 502). For example, the load instruction is included in a sequence of load instructions that have the same program counter value and access consecutive memory addresses separated by a step size. An instruction is injected into the processor's instruction stream to prefetch data using the address of the memory operation and the data of the future load instruction fetched based on the step size (block 504). In one example, the pointer array prefetcher 302 injects an instruction (e.g., a pointer target prefetch instruction 362) via the inject bus 358. The load store unit 108 receives the instruction, prefetches the data (e.g., a pointer target), and writes the prefetched data to the cache 112. Because the prefetched data has been loaded into the cache 112, the load store unit 108 can quickly provide the prefetched data from the cache 112 in response to a request for the prefetched data, e.g., a request from the execution unit 106.

[0087] It should be understood that many variations are possible based on the disclosure herein, and although features and elements are described above in specific combinations, each feature or element can be used alone without the other features and elements, or in various combinations with or without the other features and elements.

[0088] The various functional units illustrated in the figures and / or described herein (including, where appropriate, decode unit 104, execution unit 106, load store unit 108, stride prefetcher 122, pointer array prefetcher 302) may be implemented in any of a variety of different manners, such as as hardware circuits, software or firmware running on a programmable processor, or any combination of two or more of hardware, software, and firmware. The methods provided may be performed in any of a variety of devices, such as a general purpose computer, a processor, or a processor core. Suitable processors include, by way of example, a general purpose processor, a special purpose processor, a conventional processor, a Digital Signal Processor (DSP), a Graphics Processing Unit (GPU), an Accelerated Processing Unit (APU), a parallel accelerated processor, multiple microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) circuit, any other type of Integrated Circuit (IC), and / or a state machine.

[0089] In one or more embodiments, the methods provided herein may be implemented in a computer program, software, or firmware embodied in a non-transitory computer-readable storage medium for execution by a general purpose computer or processor. Examples of non-transitory computer-readable storage media include Read Only Memory (ROM), Random Access Memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and Digital Versatile Disks (DVDs).

Claims

1. 1. A method comprising: detecting a pointer target instruction by identifying a destination location of a load instruction included in a sequence of load instructions having addresses separated by a step size to be used in an address calculation of the memory operation; injecting an instruction into the processor's instruction stream to fetch data for a future load instruction using the address of the load instruction offset by a distance based on the step size. method.

2. the pointer target instruction is detected in a decode unit of the instruction stream; 2. The method of claim 1.

3. storing data of the future load instruction in a temporary register or location accessible to a decode unit of the processor.

2. The method of claim 1.

4. the memory operation is a load, a load-execute, a store, or a floating-point load; 2. The method of claim 1.

5. The distance is the product of the step size and the number of steps.

2. The method of claim 1.

6. the instruction is injected into the instruction stream via an inject bus of the processor; 2. The method of claim 1.

7. and injecting an additional instruction into the instruction stream to prefetch a pointer target based on the address of the memory operation and the data of the future load instruction.

2. The method of claim 1.

8. storing the pointer target in a cache. The method of claim 7.

9. tracking a miss rate of the L1 cache; 2. The method of claim 1.

10. detecting additional pointer target instructions based on a miss rate of the L1 cache; 10. The method of claim 9.

11. no additional pointer target instructions are detected based on the L1 cache miss rate; 10. The method of claim 9.

12. 1. A system comprising: a decode unit of the processor configured to identify a destination location of a load instruction included in a sequence of load instructions having addresses separated by a step size for use in an address calculation of a memory operation; a pointer array prefetcher of the processor configured to inject instructions into an instruction stream of the processor to prefetch data using the address of the memory operation and data of a future load instruction fetched based on the step size; system.

13. the memory operation is a load, a load-execute, a store, or a floating-point load; The system of claim 12.

14. The data is stored in an L1 cache, an L2 cache, or an L3 cache. The system of claim 12.

15. data for the future load instruction is fetched using the address of the load instruction offset by a distance based on the step size; The system of claim 12.

16. The distance is the product of the step size and the number of steps.

16. The system of claim 15.

17. the data for the future load instruction is stored in a temporary register or location accessible to the decode unit; 16. The system of claim 15.

18. 1. A method comprising: identifying a destination location of a load instruction included in a sequence of load instructions having addresses separated by a step size to be used in an address calculation of the memory operation; injecting instructions into an instruction stream of a processor to prefetch data using the address of the memory operation and data of a future load instruction fetched based on the step size. method.

19. data for the future load instruction is fetched using the address of the load instruction offset by a distance based on the step size; 20. The method of claim 18.

20. The distance is the product of the step size and the number of steps.

20. The method of claim 19.

Images