Prefetching has a skipped access pattern.

By detecting and adjusting the prefetch address of the prefetcher to accommodate jumps in access modes, the problem of useless prefetching in the prior art when processing 2D access modes is solved, thereby improving the efficiency of the processing system and CPU performance.

CN123125583APending Publication Date: 2026-11-17QUALCOMM INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202580028345.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2024-04-22
Filing Date
2025-02-27
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing prefetchers struggle to effectively detect and adjust prefetch addresses when handling 2D access patterns with jumps, leading to useless prefetches and cache misses, thus reducing the performance of the processing system.

Method used

By detecting jumps in access patterns, the prefetch address is adjusted to adapt to changes in the stride of the access patterns, and an accumulation mode is used to capture access patterns, thereby reducing cache misses.

Benefits of technology

It improves the efficiency of the processing system, reduces useless prefetching, lowers the cache miss rate, and enhances CPU performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123125583A_ABST
    Figure CN123125583A_ABST
Patent Text Reader

Abstract

Certain aspects of this disclosure provide techniques and apparatus for prefetching access patterns with skips. Aspects include obtaining the stride between consecutive memory accesses of the access pattern to determine a stride pattern for the access pattern. Aspects include determining whether a skip in the access pattern has occurred based on the stride pattern and the current stride of the access pattern. Aspects include adjusting the prefetch address for the next memory access of the access pattern based on the skip in the access pattern in response to determining that a skip has occurred. Aspects include issuing a prefetch request for the adjusted prefetch address.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Patent Application No. 18 / 642,516, filed April 22, 2024, which is incorporated herein by reference. Technical Field

[0003] All aspects of this disclosure relate to prefetchers in general, and more specifically to techniques for prefetching access patterns with jumps (such as two-dimensional (2D) access patterns having multiple rows defined by jumps). Background Technology

[0004] A processing system typically includes a central processing unit (CPU), cache memory, main memory (e.g., random access memory), and a prefetcher. The prefetcher anticipates the data (and / or instructions) the CPU may need from main memory, fetches the data from main memory, and loads the data into the cache memory. By fetching data from main memory before the CPU needs it, the prefetcher minimizes the amount of time the CPU must wait for data, thereby improving the efficiency of the processing system. Summary of the Invention

[0005] Some aspects provide a method for prefetching an access mode, the method comprising: obtaining a stride between consecutive memory accesses of the access mode to determine a stride pattern for the access mode; determining, based on the stride pattern and the current stride of the access mode, whether a jump has occurred in the access mode; in response to determining that a jump has occurred in the access mode, adjusting the prefetch address for the next memory access of the access mode based on the jump in the access mode; and issuing a prefetch request for the adjusted prefetch address.

[0006] Other aspects provide a processor including a prefetcher configured to perform the foregoing methods and the methods described herein; and a processor including components for performing the foregoing methods and the methods further described herein.

[0007] The following description and related figures illustrate certain exemplary features of one or more aspects. Attached Figure Description

[0008] The accompanying drawings depict certain features of one or more aspects of this disclosure and are therefore not intended to limit the scope of this disclosure.

[0009] Figure 1 An example computing environment for a prefetch access mode is described according to various aspects of this disclosure.

[0010] Figure 2Example access patterns with jumps are described according to various aspects of this disclosure.

[0011] Figure 3 Techniques for prefetching access patterns with skipping, according to various aspects of this disclosure, are described.

[0012] Figure 4 Methods for prefetching access patterns with skipping, according to various aspects of this disclosure, are described.

[0013] Figure 5 Methods for determining jumps in access patterns according to various aspects of this disclosure are described.

[0014] Figure 6 Example processing systems configured to perform various aspects of this disclosure are described.

[0015] For ease of understanding, the same reference numerals are used where possible to designate common elements shared by all figures. Elements and features conceived in one aspect can be beneficially incorporated into other aspects without further elaboration. Detailed Implementation

[0016] Various aspects of this disclosure provide apparatus, methods, processing systems, and computer-readable media for prefetching access patterns with skipping.

[0017] The CPU of a processing system can execute programs that include sequences of memory accesses. These sequences are often referred to as access patterns, and the prefetcher of the processing system can target specific access patterns. For example, first-class prefetchers include straddle prefetchers and stream prefetchers that target access patterns with uniform straddles. As used herein, the straddle of an access pattern can refer to the difference between memory addresses associated with consecutive memory accesses of the access pattern. Second-class prefetchers include variable-length differential prefetchers that target access patterns with repetitive differentials.

[0018] However, capturing more complex access patterns (such as 2D access patterns with multiple rows defined by jumps) can be challenging for typical prefetchers. More specifically, detecting jumps in 2D access patterns can be challenging for typical prefetchers, and thus they may continue prefetching according to the stride pattern of the 2D access pattern before a jump. This causes the prefetcher to perform prefetching of non-2D access pattern portions. These non-2D access pattern portions of the prefetch can be called useless prefetches, and these useless prefetches can lead to cache misses, degrading the performance of the processing system. As used herein, a cache miss refers to a situation where the CPU requests data that has not yet been loaded into the cache memory, and therefore the CPU must wait while the prefetcher fetches the requested data.

[0019] Certain aspects of this disclosure relate in general to techniques for prefetching access patterns with skips. More specifically, aspects of the disclosed techniques relate to detecting skips in access patterns and adjusting the prefetch address for the next memory access in the access pattern based on the detected skips. Therefore, the disclosed techniques generally allow prefetchers to capture complex access patterns more efficiently because by adjusting the prefetch address based on the detected skips in the access pattern, the disclosed techniques minimize useless prefetching (e.g., memory accesses not associated with the access pattern) that pollute the cache memory and cause cache misses (e.g., instances where the CPU-requested instruction has not yet been loaded into the cache memory), thus degrading CPU performance (e.g., efficiency).

[0020] Example computing environment for prefetching

[0021] Figure 1 An example computing environment 100 for prefetching is illustrated according to various aspects of this disclosure. The computing environment 100 includes a central processing unit (CPU) 110 configured to execute instructions to perform various computational operations. The CPU 110 may include a control unit 112 and a prefetcher 114.

[0022] Computing environment 100 includes cache memory 120 communicatively coupled to CPU 110. Cache memory 120 may store instructions 122 to be executed by CPU 110. Although cache memory 120 is depicted as separate from CPU 110, in some respects, cache memory 120 may be included as part of CPU 110.

[0023] The computing environment 100 includes main memory 130. Main memory 130 is slower than cache memory 120 and is configured to store instructions 132 to be executed by CPU 110. In some aspects, main memory 130 may include random access memory (RAM).

[0024] The prefetcher 114 of CPU 110 is configured to anticipate instructions (such as instruction 132 stored in main memory 130) that are needed by CPU 110 (such as its control unit 112) but have not yet been loaded into cache memory 120. The prefetcher 114 may also be configured to fetch instruction 132 from main memory 130 and load instruction 132 into cache memory 120 before CPU 110 needs instruction 132.

[0025] As an example, the prefetch operation performed by prefetcher 114 may include prefetcher 114 requesting instruction 132 from main memory 130 (e.g., by transmitting prefetch request 140). Prefetcher operation may include receiving instruction 132 from main memory 130 and loading instruction 132 into cache memory 120. By fetching instruction 132 from main memory 130 and loading instruction 132 into cache memory 120 before CPU 110 needs instruction 132, prefetcher 114 minimizes the amount of time CPU 110 must wait for instruction 132, thereby improving the efficiency of CPU 110.

[0026] In some aspects, the instructions 132 stored in main memory 130 may include multiple instructions stored at different memory addresses in main memory 130. For example, a first instruction for control unit 112 may be stored at a first memory address in main memory 130, and a second instruction for control unit 112 may be stored at a second memory address in main memory 130. In such aspects, prefetcher 114 may be configured to perform separate prefetch operations for the first and second instructions.

[0027] As an example, a first prefetch operation performed by prefetcher 114 may include a transfer request to read data (e.g., a first instruction) stored at a first memory address to obtain a first instruction. In this way, prefetcher 114 can obtain the first instruction to load into cache memory 120. Furthermore, a second prefetch operation performed by prefetcher 114 may include a transfer request to read data (e.g., a second instruction) stored at a second memory address to obtain a second instruction. In this way, prefetcher 114 can obtain the second instruction to load into cache memory 120.

[0028] Example access patterns with jumps

[0029] Figure 2 It is described in accordance with certain aspects of this disclosure as being able to be prefetched by a prefetcher (such as...) Figure 1 The illustrated prefetcher 114) provides a diagram of an example complex access pattern 200. As shown, access pattern 200 can be a 2D access pattern with multiple rows. The rows of access pattern 200 can be defined by jumps within access pattern 200. As shown in the diagram... Figure 2 The jump in the illustrated access mode 200 has a value of 0xE80.

[0030] Access mode 200 may begin at an initial memory address (e.g., 0x0000). This initial memory address may store data accessed by the prefetcher, or alternatively, it may simply be the starting point for the prefetcher. From this initial memory address, the prefetcher may continue to access memory addresses according to a stride pattern, which is relevant to... Figure 2 Access mode 200 is a constant stride of 0x40. However, in an alternative, the stride mode may not be a constant stride. Instead, the stride mode can be an alternating stride mode comprising multiple different strides occurring in a defined pattern. It should be understood that, as used herein, the term “stride mode” refers to an array of accumulated strides, where each stride indicates the difference between consecutive memory addresses in the access mode.

[0031] The end of each line of access mode 200 (e.g., the last memory address) may include a jump that deviates from the stride pattern of access mode 200 and represents a transition from the current line of access mode 200 to the next line of access mode 200. More specifically, the jump may correspond to the difference between the last memory address of the current line of access mode 200 and the initial memory address of the next line. As an example, a jump from the current line of access mode 200 (e.g., starting at 0x0000 and ending at 0x0180) to the next line of access mode 200 (e.g., starting at 0x1000 and ending at 0x1180) may correspond to the result of subtracting the last memory address of the current line (e.g., 0x0180) from the initial memory address of the next line (e.g., 0x1000). Therefore, for Figure 2 Access mode 200, as mentioned above, can be jumped from the current line of access mode 200 to the next line of access mode 200, which corresponds to the hexadecimal value 0x0E80.

[0032] As previously mentioned, implementations for prefetch access modes (such as...) Figure 2 The prefetcher of conventional techniques in access mode 200 generally ignores jumps in access mode 200 (e.g., +0xE80) and continues memory access according to the stride pattern of access mode 200. Figure 2 Taking access mode 200 as an example, a prefetcher implementing conventional techniques would ignore jumps (e.g., 0xE80) from the initial row of access mode 200 (e.g., ending at memory address 0x0180) to the next row of access mode 200 (e.g., starting at memory address 0x1000) and instead continue prefetching at uniform steps (e.g., 0x40). As a result, a prefetcher implementing conventional techniques for prefetching access modes would access data stored at multiple memory addresses not included in access mode 200. For example, after prefetching the data stored at memory address 0x0180, representing the end of the initial row of access mode 200, a prefetcher implementing conventional techniques for prefetching would access data at memory addresses such as 0x01C0 at uniform steps (e.g., 0x40) until the prefetcher finally reaches memory address 0x1000, representing the beginning of the next row of access mode 200.

[0033] These useless prefetches cause the prefetcher to delay accessing the data stored at the initial memory address of the next line in access mode 200, thus delaying the prefetcher's access to the CPU (e.g., Figure 1 The illustrated CPU 110 attempts to retrieve data from cache memory (e.g., ...). Figure 1 The illustrated cache memory 120 does not load data into the cache memory before fetching it. As a result, a cache miss 210 (e.g., represented by a solid black circle) occurs at the beginning of the next line in access mode 200. Furthermore, as... Figure 2 As illustrated, cache misses 210 continue to occur at the beginning of each subsequent line included in access mode 200 because the prefetcher, which implements the conventional technique used for prefetching, continues to ignore jumps in access mode 200. These cache misses 210 cause a degrade in CPU performance (e.g., efficiency) because the CPU must wait for the prefetcher to fetch the data stored at the initial memory address of the current line of access mode 200.

[0034] Example techniques for prefetching access patterns with skipping steps

[0035] Figure 3 Examples of access patterns with skipping functions, based on certain aspects of this disclosure, are illustrated. Figure 2 The technique (access mode 200) may include two modes: normal mode 300, in which jumps (e.g., 5 or more) are detected multiple times (e.g., 5 times or more) without changing the prefetch to take the jumps into account; and cumulative mode 310, in which the prefetch is modified according to the jumps detected in access mode 200 to capture access mode 200 in a more efficient manner (e.g., no cache misses). The details of normal mode 300 and cumulative mode 310 will now be discussed in detail.

[0036] While operating in normal mode 300, the stride pattern of access mode 200 is determined at least in part based on instances of a threshold number (e.g., at least two) of the stride of access mode 200. As shown, the stride of access mode 200 up to a jump in access mode 200 can be constant (e.g., 0x40) and therefore the stride pattern of access mode 200 up to a jump can be uniform (e.g., constant stride). However, in some aspects, the stride of access mode 200 up to a jump can be non-uniform (e.g., alternating between a first stride and a second stride) and therefore the stride pattern of access mode 200 up to a jump can be non-uniform.

[0037] Once the stride pattern of access mode 200 has been determined during normal mode 300, the current stride of access mode 200 can be compared with the expected stride of access mode 200 based on the determined stride pattern. If the current stride of access mode 200 matches the expected stride of access mode 200, prefetching continues according to the stride pattern. However, if the current stride of access mode 200 does not match the expected stride of access mode 200, it may be necessary to evaluate the current stride of access mode 200 to determine whether a jump has occurred in access mode 200.

[0038] When the current step size of access mode 200 deviates from the expected step size while the prefetcher is operating in normal mode 300, the prefetching is not adjusted and instead continues according to a uniform step size (e.g., 0x40) because access mode 200 cannot yet be identified as a complex access mode with defined jumps (e.g., 2D mode). Furthermore, as Figure 5 To illustrate in more detail, the disclosed techniques for prefetch access mode 200 may include verifying that various prerequisites are met to confirm a jump in access mode 200.

[0039] In some respects, a first prerequisite may be that the current step size (e.g., a jump) of access mode 200 is greater than a threshold step size. A second prerequisite may be that the current step size of access mode 200 is greater than a scaled version of the previous step size of access mode 200. More specifically, the previous step size may be the step size of the access mode immediately preceding the jump (e.g., the current step size) in access mode 200. Furthermore, the scaled version of the previous step size may be the result of multiplying the previous step size by an integer. A third prerequisite may be that the cumulative step size of access mode 200 is greater than the current step size of access mode 200, where the cumulative step size corresponds to the sum of every step size of access mode 200 up to date (e.g., including the current step size of access mode 200). A fourth prerequisite may be that both the current step size and the cumulative step size of access mode 200 are positive (e.g., greater than 0). In some respects, the fifth prerequisite may be that the difference between the cumulative step size of access mode 200 and the cumulative step size of access mode 200 immediately preceding the current step size of access mode 200 is within a threshold (e.g., less than 5 percent).

[0040] In some respects, one or more of the prerequisites mentioned above can be used heuristically. Specifically, prerequisites can be used to ensure that the current step size of access mode 200 is not out of order (e.g., the memory address is less than the most recent memory address accessed in the access mode) and therefore there are no jumps in access mode 200. For example, a third prerequisite, a fourth prerequisite, or both can be used to confirm that the current step size of access mode 200 is not out of order. If each of these prerequisites is met, a jump in access mode 200 can be confirmed.

[0041] In some aspects, if the current step size (e.g., a jump) of access mode 200 satisfies the prerequisites mentioned above, the initial instance of a jump in access mode 200 can be identified. Once the initial instance of a jump in access mode 200 is identified, the confidence variable can be incremented. Furthermore, the confidence variable can be incremented for each subsequent jump in access mode 200 that is identified. For example, the confidence variable can be initialized to 0 and incremented by 1 for each jump in the identified access mode. In some aspects, if the current step size associated with the current instance of a jump fails to satisfy one or more of the prerequisites mentioned above and therefore does not indicate a jump in access mode 200, the confidence variable can be decremented. Once an instance of the threshold number of jumps in access mode 200 (e.g., at least 5) has been identified, access mode 200 can be identified as an access mode with jumps (e.g., 2D access mode), and at this time, the prefetcher implementing the disclosed technique can switch from the normal mode 300 of the disclosed technique for performing prefetching to the cumulative mode 310 of the disclosed technique for performing prefetching, thereby capturing access mode 200 in a more efficient manner (e.g., with fewer cache misses).

[0042] As shown in the figure, entering cumulative mode 310 allows for a more efficient capture of access mode 200. For example, in cumulative mode 310, the prefetch address can be adjusted according to the cumulative step size (e.g., 0x1000) determined in access mode 200. For instance, instead of prefetching with a uniform step size of 0x40 as in normal mode 300, the prefetch performed in cumulative mode can be performed with a step size of 0x1040 (that is, the sum of 0x40 and 0x1000). In this way, the prefetcher can skip and fetch those subsequent lines of access mode 200 (e.g., the seventh line, the eighth line) before the CPU requests data to be stored at memory addresses included in subsequent lines. Therefore, cache misses (such as cache misses occurring at the beginning of each of the first few lines of access mode 200 fetched in normal mode 300) can be avoided.

[0043] In some aspects, special prefetching can be performed during cumulative mode 310. For example, such as... Figure 3 As illustrated, a stride prediction error may occur at some point during the prefetching period of access mode 200 (such as when prefetching the sixth line of access mode 200). In response to a stride prediction error, the disclosed techniques may include resetting the stride to the cumulative stride associated with a jump (e.g., 0x1000), and, while in cumulative mode 310, issuing a special prefetch at the beginning of the current line of access mode 200 and one or more subsequent lines of access mode 200.

[0044] Example methods for prefetching access patterns with skipping.

[0045] Figure 4 This is a diagram illustrating an example method 400 for prefetching access patterns with skipping according to various aspects of this disclosure. For example, method 400 may be derived from... Figure 1 The prefetcher 114 and / or the processing system described below (such as...) Figure 6 The processing system (600) is used to execute it.

[0046] Method 400 begins at block 405, where the stride between consecutive memory accesses of an access mode is obtained to determine a stride pattern for the access mode. For example, instances of a threshold number (e.g., at least two) of the stride of an access mode can be accumulated to determine the stride pattern of the access mode. The stride of an access mode may refer to the difference between memory addresses associated with consecutive memory accesses of the access mode. For example, the initial stride of an access mode may correspond to the difference between the memory address associated with the initial memory access of the access mode and the memory address associated with an additional memory access immediately following the initial memory access of the access mode. The stride of the access mode can be accumulated each time the prefetcher performs a memory access.

[0047] In some cases, the stride of the access mode up to the jump in the access mode can be constant, and therefore the stride pattern of the access mode up to the jump can be uniform (e.g., constant stride). In other cases, the stride of the access mode up to the jump can be non-uniform (e.g., alternating between the first and second strides), and therefore the stride pattern of the access mode up to the jump can be non-uniform.

[0048] Method 400 continues at box 410, wherein it is determined whether a jump has occurred in the access mode based on the current stride pattern and the access mode. In some aspects, determining whether a jump has occurred in the access mode may include comparing the current stride of the access mode with the expected stride of the access mode based on the stride pattern determined at box 405. For example, a mismatch between the current stride of the access mode and the expected stride of the access mode based on the stride pattern determined at box 405 may indicate a jump in the access mode.

[0049] Method 400 continues at block 415, wherein, in response to determining that a jump in the access mode has occurred, the prefetch address for the next memory access in the access mode is adjusted. In some aspects, the prefetch address may be adjusted to correspond to the cumulative step size of the access mode, such that the adjusted prefetch address is the memory address associated with the memory access that occurs after the jump in the access mode.

[0050] For example, the access mode could be a 2D access mode (such as...) Figure 2 and Figure 3The illustrated access mode 200 has multiple rows defined by uniform jumps, and the adjusted prefetch address may include the initial memory address on the next row of the 2D access mode. Additionally, as... Figure 3 As shown, the adjusted prefetch address may correspond to 0x1040 (e.g., indicating the sum of the cumulative step size of the jump 0x1000 and the previous step size pattern 0x40), so that the prefetcher can begin fetching data associated with the memory address included in the subsequent line of the access pattern.

[0051] Method 400 continues at block 420, wherein a prefetch is published based on the adjusted prefetch address. More specifically, method 400 is implemented to capture a prefetcher capable of performing a prefetch operation with a skipped access pattern, wherein the prefetcher fetches data stored at a memory address corresponding to the adjusted prefetch address. Additionally, the prefetcher loads the data into a cache memory (e.g., ...). Figure 1 In the illustrated cache memory 120, such that in the CPU (e.g., Figure 1 The data is available in the cache memory before the illustrated CPU 110 requests data.

[0052] Example methods for determining jumps in access patterns

[0053] Figure 5 This is a diagram illustrating an example method 500 for determining jumps in access modes according to various aspects of this disclosure. For example, method 500 may be derived from... Figure 1 The prefetcher 114 and / or the processing system described below (such as...) Figure 6 The processing system (600) is used to execute this. Furthermore, Figure 5 Method 500 can be referred to above. Figure 4 The method discussed is implemented at box 410 of 400. Furthermore, although... Figure 5 For illustrative and discussion purposes, the steps performed in a particular order are depicted, and the method 500 discussed herein is not intended to be limited to any particular order or arrangement. Those skilled in the art will understand using the disclosure provided herein that the various steps of method 500 can be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of this disclosure.

[0054] Method 500 begins at box 505, where it determines whether the current step size of the access mode is greater than a threshold step size. For example, the threshold step size could be a predefined threshold that the current step size must be greater than in order to analyze whether the current step size of the access mode is indeed a jump within the access mode. If the current step size of the access mode exceeds the threshold step size, method 500 proceeds to box 510. Otherwise, method 500 proceeds to box 515.

[0055] At box 515, method 500 includes determining whether a confidence variable associated with determining whether a jump in the access mode has occurred is greater than zero. If the confidence variable is greater than zero, the method continues to box 520, where the confidence variable is decremented, and then method 500 continues to box 525. Otherwise, method 500 skips box 520 and continues directly to box 525.

[0056] In some respects, when another prefetch operation associated with the access mode is performed and the current stride of the access mode is updated based on the most recent prefetch operation, method 500 returns to box 505 at box 525.

[0057] At box 510, the current step size of the access mode is compared with a scaled version of the previous step size of the access mode that immediately precedes the current step size. If the current step size of the access mode is greater than the scaled version of the previous step size, method 500 continues to box 530. Otherwise, the method continues to box 515.

[0058] At box 530, method 500 includes: determining whether the cumulative step size of the access mode is greater than the current step size of the access mode. As previously mentioned, the cumulative step size of the access mode corresponds to the sum of every step size of the access mode to date, including the current step size of the access mode. If the cumulative step size of the access mode is greater than the current step size of the access mode, method 500 proceeds to box 535. Otherwise, method 500 proceeds to box 515.

[0059] At box 535, method 500 includes: determining whether both the cumulative step size of the access mode and the current step size of the access mode are positive. If both the cumulative step size and the current step size are greater than zero, method 500 continues to box 540. Otherwise, method 500 continues to box 515.

[0060] At box 540, method 500 includes incrementing a confidence variable associated with determining whether a jump in the access mode has occurred to increase the confidence in method 500. Furthermore, after incrementing the confidence variable, method 500 continues to box 545.

[0061] At box 545, method 500 includes determining whether an incremented confidence variable is greater than a threshold. The threshold may be associated with the minimum confidence level required to take one or more actions in response to a jump in the acknowledged access pattern. If the incremented confidence variable is greater than the threshold, method 500 continues to box 550.

[0062] For example, such as Figure 3 As illustrated, some aspects of this disclosure may require detecting multiple instances (e.g., five or more) of the jump in access mode 200 before method 500 continues to block 550. Otherwise, method 500 continues to block 520.

[0063] At box 550, method 500 includes taking one or more actions in response to determining that a jump has occurred in the access mode. For example, the access mode could be... Figure 2 and Figure 3 The illustrated access mode 200 and one or more actions may include access from the prefetched normal mode (e.g., Figure 3 The illustrated normal mode 300) to the prefetched cumulative mode (e.g., Figure 3 The illustrated cumulative mode 310 is transformed to capture access mode 200 in a more efficient manner than conventional prefetching techniques (e.g., with fewer cache misses). In cumulative mode, the prefetch address used for subsequent prefetching can be adjusted based on the cumulative step size (e.g., 0x1000) indicating a jump in the identified access mode 200. In this way, the implementation... Figure 5 Method 500's prefetcher can improve CPU efficiency because the prefetcher can more efficiently fetch complex access patterns with jumps (such as...) in a way that avoids cache misses at the beginning of each line (e.g., no cache misses at the beginning of each line). Figure 2 Access mode 200).

[0064] Example processing system for prefetching access patterns with skipping.

[0065] In some aspects, refer to Figures 3 to 5 The described techniques and methods can be implemented on one or more devices or systems. Figure 6 An example processing system 600 is depicted, which is configured to perform various aspects of this disclosure, including, for example, regarding... Figures 3 to 5 The techniques and methods described. In some aspects, the processing system 600 may correspond to... Figure 1 The computing environment 100. Although depicted as a single system for clarity of concept, in some respects, as discussed above, the operations described below with respect to processing system 600 can be distributed across any number of devices or systems.

[0066] Processing system 600 includes a central processing unit (CPU) 602 (e.g., corresponding to...). Figure 1 The instructions executed at CPU 602 may, for example, be loaded from cache memory 626 associated with CPU 602 (e.g., corresponding to CPU 110). Figure 1 Cache memory 120).

[0067] The processing system 600 also includes additional processing components tailored for specific functions, such as a graphics processing unit (GPU) 604, a digital signal processor (DSP) 606, a neural processing unit (NPU) 608, a multimedia component 610 (e.g., a multimedia processing unit), and a wireless connectivity component 612.

[0068] NPUs (such as the NPU 608) are typically configured to implement dedicated circuitry for implementing control and arithmetic logic for executing machine learning algorithms, such as those for processing artificial neural networks (ANNs), deep neural networks (DNNs), random forests (RFs), etc. NPUs are sometimes alternatively referred to as neural signal processors (NSPs), tensor processing units (TPUs), neural network processors (NNPs), intelligent processing units (IPUs), vision processing units (VPUs), or graphics processing units.

[0069] NPUs (such as the NPU 608) are configured to accelerate the execution of common machine learning tasks, such as image classification, machine translation, object detection, and various other predictive models. In some examples, multiple NPUs may be instantiated on a single chip, such as a SoC, while in other examples, an NPU may be part of a dedicated neural network accelerator.

[0070] An NPU can be optimized for either training or inference, or in some cases configured to balance performance between the two. For an NPU capable of performing both training and inference, these two tasks can generally still be performed independently.

[0071] NPUs designed to accelerate training are typically configured to speed up the optimization of new models. This involves taking an existing dataset (usually labeled or sublabeled), iterating over the dataset, and subsequently tuning model parameters (such as weights and biases) to improve model performance—a highly computationally intensive operation. Generally, optimization based on incorrect predictions involves backtracking through the layers of the model and determining gradients to reduce prediction errors.

[0072] NPUs designed to accelerate inference are typically configured to operate on the full model. Therefore, such NPUs can be configured to take new data segments as input and rapidly process those segments through a trained model to generate model outputs (e.g., inference).

[0073] In some implementations, the NPU 608 is part of one or more of the CPU 602, GPU 604, and / or DSP 606.

[0074] In some examples, the wireless connectivity component 612 may include sub-components for, for example, third-generation (3G) connectivity, fourth-generation (4G) connectivity (e.g., 4G Long Term Evolution (LTE)), fifth-generation connectivity (e.g., 5G or New Radio (NR)), Wi-Fi connectivity, Bluetooth connectivity, and / or other wireless data transmission standards. The wireless connectivity component 612 is further coupled to one or more antennas 614.

[0075] The processing system 600 may also include one or more sensor processing units 616 associated with any type of sensor, one or more image signal processors (ISPs) 618 associated with any type of image sensor, and / or a navigation processor 620, which may include satellite-based positioning system components (e.g., GPS or GLONASS) and inertial positioning system components.

[0076] The processing system 600 may also include one or more input and / or output devices 622, such as screens, touch-sensitive surfaces (including touch-sensitive displays), physical buttons, speakers, microphones, etc.

[0077] In some examples, one or more processors in the processing system 600 may be based on the ARM or RISC-V instruction set.

[0078] The processing system 600 also includes a memory 624, which represents one or more static and / or dynamic memories, such as dynamic random access memory, flash-based static memory, etc. In this example, the memory 624 includes computer-executable components that can be executed by one or more of the aforementioned processors of the processing system 600.

[0079] Memory 624 may include cache memory 626 (e.g., corresponding to...) Figure 1 The illustrated cache memory 120). Memory 624 may also include main memory 628 (e.g., corresponding to...). Figure 1 The illustrated main memory 130. Cache memory 626 may include instructions 630 to be executed by CPU 602. Main memory 628 also includes instructions 632 to be executed by CPU 602. As previously discussed, a prefetcher (e.g., Figure 1 The illustrated prefetcher 114 can predict the CPU 602's need for instructions 632 from main memory 628 and can execute techniques (such as...) Figure 4 and Figure 5 The method is to fetch instruction 632 from main memory 628 and load instruction 632 into cache memory 626 before CPU 602 requests instruction 632.

[0080] Generally speaking, the processing system 600 and / or its components can be configured to perform the methods described herein.

[0081] It is worth noting that, in other aspects, such as when the processing system 600 is a server computer, components of the processing system 600 may be omitted. For example, in other aspects, the multimedia component 610, the wireless connectivity component 612, the sensor processing unit 616, the ISP 618, and / or the navigation processor 620 may be omitted. Furthermore, various aspects of the processing system 600 may be distributed among multiple devices.

[0082] Example Terms

[0083] Specific implementation examples are described in the following numbered clauses: Clause 1: A method for a prefetch access mode, the method comprising: obtaining a stride between consecutive memory accesses of the access mode to determine a stride pattern for the access mode; determining, based on the stride pattern and a current stride of the access mode, whether a jump has occurred in the access mode; in response to determining that a jump has occurred in the access mode, adjusting a prefetch address for the next memory access of the access mode based on the jump in the access mode; and issuing a prefetch request for the adjusted prefetch address.

[0084] Clause 2: The method according to Clause 1, wherein determining that the jump in the access mode has occurred includes: determining that the current step size of the access mode is greater than a threshold step size.

[0085] Clause 3: The method according to Clause 2, wherein determining that the jump in the access mode has occurred further comprises: determining that the current stride is greater than a scaled version of the stride immediately preceding the current stride in the access mode.

[0086] Clause 4: The method according to Clause 2 or 3, wherein determining that the jump in the access mode has occurred further includes: determining whether the cumulative step size of the access mode is greater than the current step size of the access mode.

[0087] Clause 5: The method according to any one of Clauses 2 to 4, wherein determining that the jump in the access mode has occurred further includes: determining that the cumulative step size is positive; and determining that the current step size is positive.

[0088] Clause 6: The method according to any one of Clauses 1 to 5, wherein the cumulative step size includes the current step size; and adjusting the prefetch address for the next memory access of the access mode includes adding the cumulative step size to the memory address associated with the initial memory access of the access mode.

[0089] Clause 7: The method according to any one of Clauses 1 to 6, the method further comprising: in response to determining that the jump in the access mode has occurred, incrementing a confidence variable to indicate an increase in confidence for detecting the jump in the access mode.

[0090] Clause 8: The method according to Clause 7 further includes: determining whether the confidence variable is greater than a threshold; and adjusting the prefetch address for the next memory access of the access mode in response to determining that the confidence variable is greater than the threshold.

[0091] Clause 9: The method according to Clause 7 further includes: decrementing the confidence variable in response to determining that the jump in the access mode has not yet occurred.

[0092] Clause 10: The method according to any one of Clauses 1 to 9, wherein: the access mode includes a two-dimensional access mode having a plurality of rows; and the jump indicates a transition from a first row of the plurality of rows to a second row of the plurality of rows immediately below the first row.

[0093] Clause 11: The method according to Clause 10, wherein adjusting the prefetch address for the next memory access for the access mode comprises: adjusting the prefetch address to the memory address at the beginning of the second row.

[0094] Clause 12: A processor comprising: a prefetcher configured to execute computer-executable instructions to cause the prefetcher to: obtain a stride between consecutive memory accesses of an access mode to determine a stride pattern for the access mode; determine, based on the stride pattern and a current stride of the access mode, whether a jump has occurred in the access mode; in response to determining that a jump has occurred in the access mode, adjust a prefetch address for the next memory access of the access mode based on the jump in the access mode; and issue a prefetch request for the adjusted prefetch address.

[0095] Clause 13: The processor according to Clause 12, wherein in order to determine that the jump in the access mode has occurred, the prefetcher is configured to determine that the current step size of the access mode is greater than a threshold step size.

[0096] Clause 14: The processor according to Clause 13, wherein in order to determine that the jump in the access mode has occurred, the prefetcher is configured to determine that the current stride is greater than a scaled version of the stride immediately preceding the current stride in the access mode.

[0097] Clause 15: A processor according to Clause 13 or 14, wherein, in order to determine that the jump in the access mode has occurred, the prefetcher is configured to determine that the cumulative step size of the access mode is greater than the current step size of the access mode.

[0098] Clause 16: A processor according to any one of Clauses 13 to 15, wherein, in order to determine that the jump in the access mode has occurred, the prefetcher is configured to determine that the cumulative step size is positive and to determine whether the current step size is positive.

[0099] Clause 17: A processor according to any one of Clauses 13 to 16, wherein: the cumulative step size includes the current step size; and in order to adjust the prefetch address for the next memory access of the access mode, the prefetcher is configured to add the cumulative step size to the memory address associated with the initial memory access of the access mode.

[0100] Clause 18: A processor according to any one of Clauses 12 to 17, wherein the prefetcher is configured to: in response to determining that the jump in the access mode has occurred, increment a confidence variable to indicate an increase in confidence for detecting the jump in the access mode.

[0101] Clause 19: The processor according to Clause 18, wherein the prefetcher is further configured to: determine whether the confidence variable is greater than a threshold; and, in response to determining that the confidence variable is greater than the threshold, adjust the prefetch address for the next memory access of the access mode.

[0102] Clause 20: An apparatus comprising: means for obtaining a stride between consecutive memory accesses of an access mode to determine a stride pattern for the access mode; means for determining, based on the stride pattern and a current stride of the access mode, whether a jump has occurred in the access mode; means for adjusting a prefetch address for the next memory access of the access mode based on the jump in the access mode in response to determining that a jump has occurred in the access mode; and means for issuing a prefetch request for the adjusted prefetch address.

[0103] Additional Notes

[0104] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. The examples discussed herein do not limit the scope, applicability, or aspects set forth in the claims. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. For example, the function and arrangement of the elements discussed may be changed without departing from the scope of this disclosure. Various processes or components may be omitted, substituted, or added as appropriate in various examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined in some other examples. For example, any number of aspects set forth herein may be used to implement an apparatus or practice. Moreover, the scope of this disclosure is intended to cover such apparatuses or methods practiced using other structures, functionalities, or structures and functionalities that complement or replace the various aspects of this disclosure set forth herein. It should be understood that any aspect of this disclosure disclosed herein may be embodied by one or more elements of these claims.

[0105] For example, components used to determine the stride pattern for access patterns by obtaining the stride between consecutive memory accesses may include a prefetcher (e.g., such as...). Figure 1 The illustrated pointer prefetcher 114). Components for determining whether a jump in the access mode has occurred based on the current stride of the stride mode and the access mode may include a prefetcher. Components for adjusting the prefetch address for the next memory access in the access mode based on the jump in the access mode may include a prefetcher. Components for issuing a prefetch request for the adjusted prefetch address may include a prefetcher.

[0106] As used herein, the term “exemplary” means “serving as an example, instance, or illustration.” Any aspect described as “exemplary” in this document is not necessarily to be construed as preferred or superior to other aspects.

[0107] As used in this article, the phrase “at least one of” in a list of items refers to any combination of these items, including a single member. For example, “at least one of a, b, or c” is intended to cover a, b, c, ab, ac, bc, and abc, as well as any combination with multiple identical elements (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc, or any other ordering of a, b, and c).

[0108] As used herein, the term "determine" encompasses a wide variety of actions. For example, "determine" can include calculation, computation, processing, derivation, research, searching (e.g., searching in a table, database, or other data structure), ascertainment, and similar actions. Additionally, "determine" can include receiving (e.g., receiving information), accessing (e.g., accessing memory), etc. Furthermore, "determine" can include parsing, selecting, picking, building, etc.

[0109] The methods disclosed herein include one or more steps or actions for implementing the methods. The steps and / or actions of the methods may be interchanged without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of a particular step and / or action may be modified without departing from the scope of the claims. Furthermore, the various operations of the methods described above may be performed by any suitable component capable of performing the corresponding function. This component may include various hardware and / or software components and / or modules, including but not limited to circuits, application-specific integrated circuits (ASICs), or processors. Generally, where the operations illustrated in the figures are present, those operations may have corresponding components with similar numbering plus functional components.

[0110] The following claims are not intended to be limited to the aspects shown herein, but should be given the full scope consistent with the language of the claims. Within the claims, unless specifically stated otherwise, reference to the singular form of an element is not intended to mean “one and only one,” but rather “one or more.” Unless specifically stated otherwise, the term “some” refers to one or more. No element of any claim should be interpreted in accordance with 35 USC § 112(f) unless the element is expressly referred to by the phrase “for a component of,” or, in the case of a method claim, by the phrase “for a step of.” All structural and functional equivalents of the elements throughout the various aspects described herein that are known to a person of ordinary skill in the art, or will be known later, are expressly incorporated herein by reference and are intended to be covered by the claims. Furthermore, nothing disclosed herein is intended to be offered to the public, whether or not such disclosure is explicitly stated in the claims.

Claims

1. A method for a prefetch access mode, the method comprising: The stride between consecutive memory accesses of the access mode is obtained to determine the stride pattern for the access mode; Determine whether a jump has occurred in the access mode based on the current stride pattern and the access mode; In response to determining that a jump has occurred in the access mode, the prefetch address for the next memory access in the access mode is adjusted based on the jump in the access mode; as well as A prefetch request is issued for the adjusted prefetch address.

2. The method of claim 1, wherein determining that the jump has occurred in the access mode comprises: It is determined that the current step size of the access mode is greater than the threshold step size.

3. The method of claim 2, wherein determining that the jump has occurred in the access mode further comprises: Determine a scaled version of the current stride that immediately precedes the current stride in the access mode.

4. The method of claim 2, wherein determining that the jump has occurred in the access mode further comprises: Determine whether the cumulative step size of the access mode is greater than the current step size of the access mode.

5. The method of claim 2, wherein determining that the jump has occurred in the access mode further comprises: The cumulative step size of the access mode is determined to be positive; as well as The current stride is determined to be positive.

6. The method according to claim 1, wherein: The cumulative step size of the access mode includes the current step size; and Adjusting the prefetch address for the next memory access of the access mode includes adding the cumulative step size to the memory address associated with the initial memory access of the access mode.

7. The method according to claim 1, further comprising: In response to determining that the jump in the access mode has occurred, the confidence variable is incremented to indicate an increase in confidence for detecting the jump in the access mode.

8. The method according to claim 7, further comprising: Determine whether the confidence variable is greater than the threshold; as well as In response to determining that the confidence variable is greater than the threshold, the prefetch address for the next memory access in the access mode is adjusted.

9. The method according to claim 7, further comprising: In response to determining that the jump in the access mode has not yet occurred, the confidence variable is decremented.

10. The method according to claim 1, wherein: The access mode includes a two-dimensional access mode, which has multiple rows; and The jump indicates a transition from the first row of the plurality of rows to the second row of the plurality of rows, the second row immediately following the first row.

11. The method of claim 10, wherein adjusting the prefetch address for the next memory access in the access mode comprises: Adjust the prefetch address to the memory address at the beginning of the second row.

12. A processor, the processor comprising: A prefetcher, the prefetcher being configured to execute computer-executable instructions to cause the prefetcher to: The stride between consecutive memory accesses of the access mode is obtained to determine the stride pattern for the access mode; Determine whether a jump has occurred in the access mode based on the current stride pattern and the access mode; In response to determining that a jump has occurred in the access mode, the prefetch address for the next memory access in the access mode is adjusted based on the jump in the access mode; as well as A prefetch request is issued for the adjusted prefetch address.

13. The processor of claim 12, wherein, in order to determine that the jump in the access mode has occurred, the prefetcher is configured to determine that the current step size of the access mode is greater than a threshold step size.

14. The processor of claim 13, wherein, in order to determine that the jump in the access mode has occurred, the prefetcher is configured to determine that the current stride is greater than a scaled version of the stride immediately preceding the current stride in the access mode.

15. The processor of claim 13, wherein, in order to determine that the jump in the access mode has occurred, the prefetcher is configured to determine that the cumulative step size of the access mode is greater than the current step size of the access mode.

16. The processor of claim 13, wherein, in order to determine that the jump in the access mode has occurred, the prefetcher is configured to determine that the cumulative step size of the access mode is positive and to determine that the current step size is positive.

17. The processor according to claim 13, wherein: The cumulative step size of the access mode includes the current step size of the access mode; and In order to adjust the prefetch address for the next memory access of the access mode, the prefetcher is configured to add the cumulative step size to the memory address associated with the initial memory access of the access mode.

18. The processor of claim 12, wherein the prefetcher is configured to: In response to determining that the jump in the access mode has occurred, the confidence variable is incremented to indicate an increase in confidence for detecting the jump in the access mode.

19. The processor of claim 18, wherein the prefetcher is further configured to: Determine whether the confidence variable is greater than the threshold; and In response to determining that the confidence variable is greater than the threshold, the prefetch address for the next memory access in the access mode is adjusted.

20. An apparatus comprising: Components for determining the stride pattern for the access mode by obtaining the stride between consecutive memory accesses of the access mode; A component for determining whether a jump has occurred in the access mode based on the current stride of the stride mode and the access mode; A component for adjusting the prefetch address for the next memory access in the access mode based on the jump in the access mode in response to determining that the jump in the access mode has occurred; and A component used to issue prefetch requests for the adjusted prefetch address.