Data prefetching method and apparatus, computer device, and storage medium
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-10-13
- Publication Date
- 2026-08-13
Smart Images
Figure CN2025127408_13082026_PF_FP_ABST
Abstract
Description
Data prefetching methods, apparatus, computer equipment, and storage media
[0001] Cross-references to related applications
[0002] This disclosure is based on and claims priority to Chinese Patent Application No. 202510138753.7, filed on February 8, 2025, the entire contents of which are incorporated herein by reference. Technical Field
[0003] This disclosure relates to the field of processor technology, and more specifically to a data prefetching method, apparatus, computer device, and storage medium. Background Technology
[0004] The development of processor storage speed has lagged far behind the development of processor computing speed, and the increasingly severe "memory wall" problem seriously restricts further improvements in processor performance. Cache is a storage mechanism designed using the principle of locality of access, aiming to bridge the huge gap between processor computing speed and memory access speed as much as possible. The "memory wall" problem has driven the development of caches towards larger capacity and multi-level structures. While multi-level cache structures can reduce the average latency of data accesses with good locality, they cannot reduce the latency of first memory access, i.e., compulsory misses. Summary of the Invention
[0005] This disclosure aims to at least partially address one of the technical problems in the related art.
[0006] Therefore, the purpose of this disclosure is to provide a data prefetching method, apparatus, computer device, and storage medium.
[0007] To achieve the above objectives, the data prefetching method proposed in the first aspect of this disclosure, applied to a processor, includes:
[0008] After receiving the prefetch mode and prefetch distance from the first-level storage structure, the nth-level storage structure of the processor determines the target data address from the multiple initial data addresses included in the prefetch mode according to the prefetch distance, where n is an integer greater than or equal to 1, and the prefetch distance is used to indicate the number of initial data addresses for which the nth-level storage structure does not perform data prefetching.
[0009] The nth level storage structure sends a prefetch request corresponding to each target data address to the (n+1)th level storage structure.
[0010] To achieve the above objectives, a data prefetching apparatus according to a second aspect embodiment of this disclosure, applied to a processor, includes:
[0011] A determining module is used to determine a target data address from a plurality of initial data addresses contained in the prefetch mode according to the prefetch distance after the nth level memory structure of the processor receives the prefetch mode and prefetch distance of the first level memory structure, wherein n is an integer greater than or equal to 1, and the prefetch distance is used to indicate the number of initial data addresses for which the nth level memory structure does not perform data prefetching.
[0012] The sending module is used to send a prefetch request corresponding to each target data address from the nth level storage structure to the (n+1)th level storage structure.
[0013] A computer device according to a third aspect of this disclosure includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a data prefetching method as proposed in a first aspect of this disclosure.
[0014] A fourth aspect of this disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform a data prefetching method as described in a first aspect of this disclosure.
[0015] A fifth aspect of this disclosure provides a computer program product in which, when instructions in the computer program product are executed by a processor, a data prefetching method as described in a first aspect of this disclosure is performed.
[0016] This disclosure provides a data prefetching method, apparatus, computer device, and storage medium. The method includes: after receiving a prefetching mode and prefetching distance from a first-level storage structure, the nth-level storage structure of the processor determines a target data address from a plurality of initial data addresses included in the prefetching mode based on the prefetching distance, where n is an integer greater than or equal to 1, and the prefetching distance indicates the number of initial data addresses for which the nth-level storage structure does not perform data prefetching; the nth-level storage structure sends a prefetch request corresponding to each target data address to the (n+1)th-level storage structure. This effectively improves the timeliness of prefetching, reduces cache pollution caused by prefetching, and thus improves the overall performance of the processor.
[0017] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description
[0018] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which:
[0019] Figure 1 is a schematic flowchart of a data prefetching method proposed in an embodiment of this disclosure;
[0020] Figure 2 is a schematic flowchart of a data prefetching method proposed in another embodiment of this disclosure;
[0021] Figure 3 is a schematic flowchart of a data prefetching method proposed in another embodiment of this disclosure;
[0022] Figure 4 is a schematic diagram of the memory hierarchy proposed according to this disclosure;
[0023] Figure 5 is a diagram of the multi-level timeliness prefetching architecture proposed in this disclosure;
[0024] Figure 6 is a diagram of the prefetch components of each level of cache according to the present disclosure;
[0025] Figure 7 is a schematic diagram of the hardware implementation of the spatial storage stream prefetching strategy proposed in this disclosure;
[0026] Figure 8 is a schematic diagram of the structure of a data prefetching device according to an embodiment of the present disclosure;
[0027] Figure 9 shows a block diagram of an exemplary computer device suitable for implementing embodiments of the present disclosure. Detailed Implementation
[0028] Embodiments of this disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are used only to explain this disclosure, and should not be construed as limiting this disclosure. Rather, embodiments of this disclosure include all variations, modifications, and equivalents falling within the spirit and scope of the appended claims.
[0029] A multi-level cache structure can reduce the average latency of data access with good locality, but it cannot reduce the latency of the first memory access, i.e., the computational miss.
[0030] Prefetching is crucial for improving processor memory access performance. A key way to improve memory access performance is to reduce cache miss rate. Prefetching reduces computational misses by retrieving cache lines that the processor might need in advance from different memory levels. Essentially, prefetching is a guessing mechanism for program memory access addresses, requiring analysis of program behavior. Its core principle revolves around leveraging spatial and temporal locality during program execution, and its performance improvement is highly correlated with the characteristics of the executed program. If the prefetched data block hasn't returned when the needed data block is used, resulting in untimely prefetching, it reduces the performance improvement effect of prefetching. Prefetching consumes processor memory bandwidth and generates additional power consumption. Furthermore, fetching incorrect cache lines or using the wrong prefetch timing can pollute the cache. Prefetching implementation requires careful consideration; problems with prefetching can not only fail to improve overall processor performance but may actually degrade it.
[0031] Among the related technologies, there is no effective solution to the problems of untimely prefetching and cache pollution caused by prefetching.
[0032] Figure 1 is a schematic flowchart of a data prefetching method proposed in an embodiment of this disclosure.
[0033] It should be noted that the execution subject of the data prefetching method in this embodiment is a data prefetching device, which can be implemented by software and / or hardware. The device can be configured in a computer device, which may include, but is not limited to, a terminal, a server, etc., such as a mobile phone, a PDA, etc.
[0034] As shown in Figure 1, this data prefetching method includes:
[0035] S101: After receiving the prefetch mode and prefetch distance from the first-level memory structure, the nth-level memory structure of the processor determines the target data address from the multiple initial data addresses contained in the prefetch mode according to the prefetch distance, where n is an integer greater than or equal to 1, and the prefetch distance is used to indicate the number of initial data addresses for which the nth-level memory structure does not perform data prefetching.
[0036] The processor's memory structure refers to the structure configured within the processor for storing data. For example, the processor's memory structure may include an on-chip N-level cache.
[0037] The first level of storage structure can refer to the L1 Cache, which is the closest cache level to the processor core in the N-level cache.
[0038] The prefetch mode can be used to indicate the address of the data to be prefetched.
[0039] The prefetch distance refers to the number of initial data addresses in the nth level storage structure that are not prefetched.
[0040] The initial data address refers to the data address contained in the prefetch pattern. The target data address refers to the address at which the nth level storage structure needs to prefetch data from the (n+1)th level storage structure.
[0041] It is understood that the multiple initial data addresses contained in the prefetch pattern are arranged in order. In this embodiment of the present disclosure, determining the target data address from the multiple initial data addresses contained in the prefetch pattern based on the prefetch distance may include: ignoring a specified number of initial data addresses that are ranked first based on the prefetch distance, and taking the initial data addresses ranked last as the target data address.
[0042] S102: The nth level storage structure sends a prefetch request corresponding to each target data address to the (n+1)th level storage structure.
[0043] Here, a prefetch request refers to a request sent by the nth level storage structure to the (n+1)th level storage structure to prefetch the data corresponding to the target data address.
[0044] It is understandable that a prefetch request sent from the nth level storage structure to the (n+1)th level storage structure can be regarded as a data access request.
[0045] In some embodiments, sending a prefetch request corresponding to each target data address from the nth level storage structure to the (n+1)th level storage structure may include: determining a first matching result between the prefetch request and a historical prefetch record table, wherein the historical prefetch record table is used to record prefetch requests sent by the nth level storage structure to the (n+1)th level storage structure; and when the first matching result indicates that the nth level storage structure has not sent a prefetch request to the (n+1)th level storage structure, the nth level storage structure sends a prefetch request to the (n+1)th level storage structure. This effectively avoids the bandwidth and power consumption loss caused by the nth level storage structure sending prefetch requests multiple times to the same data address.
[0046] The historical prefetch record table refers to a pre-configured table used to record prefetch requests sent from the nth level storage structure to the (n+1)th level storage structure.
[0047] The first matching result can be used to indicate whether the aforementioned prefetch request is contained in the historical prefetch record table.
[0048] That is to say, in this embodiment of the present disclosure, when the first matching result indicates that the nth level storage structure has sent a prefetch request to the (n+1)th level storage structure, the nth level storage structure can ignore the corresponding prefetch request.
[0049] In the embodiments of this disclosure, after receiving the prefetch mode and prefetch distance from the first-level storage structure, the nth-level storage structure of the processor determines the target data address from a plurality of initial data addresses included in the prefetch mode based on the prefetch distance, where n is an integer greater than or equal to 1, and the prefetch distance is used to indicate the number of initial data addresses for which the nth-level storage structure does not perform data prefetching; the nth-level storage structure sends a prefetch request corresponding to each target data address to the (n+1)th-level storage structure. This effectively improves the timeliness of prefetching, reduces cache pollution caused by prefetching, and thus improves the overall performance of the processor.
[0050] Figure 2 is a schematic flowchart of a data prefetching method proposed in another embodiment of this disclosure.
[0051] As shown in Figure 2, this data prefetching method includes:
[0052] S201: After receiving the first access request from the corresponding upper-level storage structure, the nth-level storage structure determines the second matching result between the first access request and the historical prefetch record table. The historical prefetch record table is used to record the prefetch requests that the nth-level storage structure has sent to the (n+1)th-level storage structure. The upper-level storage structure corresponding to the first-level storage structure is the processor core of the processor.
[0053] The next level of storage structure can refer to a storage structure that is closer to the processor core than the nth level of storage structure.
[0054] Here, the first access request refers to a request sent by the upper-level storage structure to the nth-level storage structure for data access. In this embodiment of the disclosure, the first access request includes a prefetch request sent by the upper-level storage structure to the nth-level storage structure.
[0055] The second matching result can be used to indicate whether the nth level storage structure has sent a prefetch request with the same target data address as the first access request to the (n+1)th level storage structure.
[0056] In other words, in this embodiment of the present disclosure, after receiving the first access request from the corresponding upper-level storage structure, the nth-level storage structure can first perform matching processing on the first access request based on the historical prefetch record table to determine whether the nth-level storage structure has sent a prefetch request with the same target data address as the first access request to the n+1th-level storage structure.
[0057] S202: When the nth level storage structure meets the preset conditions, the prefetch distance is updated, wherein the preset conditions include: the second matching result indicates that the nth level storage structure has sent a prefetch request with the same target data address as the first access request to the (n+1)th level storage structure, and there is no data in the nth level storage structure corresponding to the first access request.
[0058] The preset conditions refer to the conditions configured in this embodiment for whether the prefetch distance of the nth level storage structure needs to be updated. The specific content of these preset conditions can be flexibly configured according to the application scenario, and there are no restrictions on them.
[0059] In some embodiments, updating the prefetch distance may include: determining the prefetch performance metrics of the nth level storage structure; and updating the prefetch distance based on the prefetch performance metrics. Therefore, by combining the prefetch performance metrics of the nth level storage structure, the practicality and effectiveness of the updated prefetch distance can be guaranteed.
[0060] Among them, the prefetch performance metric can be used to measure the data prefetching capability of the nth level storage structure.
[0061] In some embodiments, the prefetch performance metrics include at least one of the following: the prefetch hit rate of the nth level storage structure; and the number of times the nth level storage structure continuously meets preset conditions. This ensures that the prefetch performance metrics accurately indicate the data prefetching capability of the nth level storage structure.
[0062] Prefetch hit rate can be used to indicate the number of successful prefetches in multiple data prefetching processes.
[0063] In the embodiments of this disclosure, after receiving a first access request from the corresponding parent storage structure, the nth-level storage structure determines a second matching result between the first access request and a historical prefetch record table. The historical prefetch record table records prefetch requests sent by the nth-level storage structure to the (n+1)th-level storage structure. The parent storage structure corresponding to the first-level storage structure is the processor core. When preset conditions are met, the nth-level storage structure updates the prefetch distance. The preset conditions include: the second matching result indicating that the nth-level storage structure has sent a prefetch request with the same target data address as the first access request to the (n+1)th-level storage structure, and there is no data corresponding to the first access request in the nth-level storage structure. Therefore, flexible adjustment of the prefetch distance can be achieved to ensure that the updated prefetch distance is applicable to diverse application scenarios.
[0064] In some embodiments, determining the target data address from a plurality of initial data addresses included in the prefetch pattern based on the prefetch distance may include: determining the target data address from a plurality of initial data addresses included in the prefetch pattern based on an updated prefetch distance. Therefore, the usability of the obtained target data address can be effectively improved based on the updated prefetch distance.
[0065] Figure 3 is a schematic flowchart of a data prefetching method proposed in another embodiment of this disclosure.
[0066] As shown in Figure 3, this data prefetching method includes:
[0067] S301: The processor's Level 1 memory structure receives a second access request from the processor core and generates a prefetch mode based on the second access request.
[0068] In other words, in this embodiment of the present disclosure, the first-level storage structure of the processor can receive the second access request from the processor core and generate a prefetch mode according to the second access request, thereby providing a reliable execution basis for the prefetching process of subsequent storage structures.
[0069] In some embodiments, generating a prefetch pattern based on a second access request may include: determining a target event identifier corresponding to the second access request; determining a historical event identifier matching the target event identifier from a pattern history table, wherein the pattern history table includes multiple historical event identifiers and a historical pattern corresponding to each historical event identifier; and using the historical pattern corresponding to the matching historical event identifier as the prefetch pattern. Therefore, by combining the processor's historical data access information, a suitable prefetch pattern can be quickly and accurately determined, ensuring the practicality of the obtained prefetch pattern.
[0070] S302: The first-level storage structure determines the prefetch distance corresponding to the nth-level storage structure and sends the prefetch mode and prefetch distance to the nth-level storage structure.
[0071] In this embodiment of the disclosure, when determining the prefetch distance corresponding to the nth level storage structure, it can be based on a preset relationship table containing the prefetch distance corresponding to each storage structure, or it can be based on a data combination method to determine the prefetch distance corresponding to the nth level storage structure, without limitation.
[0072] In some embodiments, determining the prefetch distance corresponding to the nth level storage structure may include: determining the number of storage structure layers separating the first level storage structure from the nth level storage structure; determining the initial distance corresponding to the nth level storage structure based on the number of storage structure layers; and determining the prefetch distance corresponding to the nth level storage structure based on the initial distance. Therefore, the applicability of the obtained prefetch distance can be ensured by combining the location information of different storage structures.
[0073] The number of storage structure layers can be used to indicate the number of storage structure layers between the first-level storage structure and the nth-level storage structure. For example, the number of storage structure layers between the first-level storage structure and the seventh-level storage structure is 6.
[0074] The initial distance can refer to the prefetch distance in the initial state of the nth level storage structure, based on the number of storage structure layers.
[0075] For example, in this embodiment of the present disclosure, when determining the initial distance corresponding to the nth level storage structure based on the number of storage structure layers, it can be based on a pre-configured relation table containing the initial distance corresponding to the number of storage structure layers. Alternatively, it can be based on statistical or intelligent learning methods to determine the initial distance corresponding to the nth level storage structure based on the number of storage structure layers. There are no restrictions on this.
[0076] In some embodiments, determining the initial distance corresponding to the nth level of storage structure based on the number of storage structure layers may include: determining the product of a preset distance value and the number of storage structure layers as the initial distance. Therefore, the initial distance corresponding to different levels of storage structure can be quickly configured based on the preset distance value.
[0077] The preset distance value can be flexibly configured according to the application scenario, and there are no restrictions on it. For example, when the prefetch distance value is configured as 2, the initial distance corresponding to the 5th level storage structure is 2*(5-1)=8.
[0078] In this embodiment of the disclosure, when determining the prefetch distance corresponding to the nth level storage structure based on the initial distance, the initial distance can be directly used as the prefetch distance corresponding to the nth level storage structure, or a third-party device can adjust the initial distance to obtain the prefetch distance corresponding to the nth level storage structure. There are no restrictions on this.
[0079] In some embodiments, determining the prefetch distance corresponding to the nth level storage structure based on the initial distance may include: determining the prefetch performance metrics of the nth level storage structure during multiple data prefetching processes; and adjusting the initial distance based on multiple prefetch performance metrics to obtain the prefetch distance corresponding to the nth level storage structure. Therefore, by combining the prefetch performance metrics of the nth level storage structure during multiple data prefetching processes, the data prefetching capability of the nth level storage structure can be evaluated, and the initial distance can be adaptively adjusted to ensure that the obtained preset distance can effectively improve the data prefetching capability of the nth level storage structure.
[0080] For example, in this embodiment of the disclosure, determining the prefetch distance corresponding to the nth level of the storage structure based on the number of storage structure layers may include: configuring the specific value of the prefetch distance to be proportional to the number of storage structure layers.
[0081] In the embodiments of this disclosure, the first-level memory structure of the processor receives a second access request from the processor core and generates a prefetch pattern based on the second access request; the first-level memory structure determines the prefetch distance corresponding to the nth-level memory structure and sends the prefetch pattern and prefetch distance to the nth-level memory structure. Therefore, the reliability of the prefetch pattern and prefetch distance obtained by the nth-level memory structure can be guaranteed.
[0082] In conjunction with the above embodiments, this disclosure addresses the main problems of untimely prefetching and cache pollution caused by prefetching. By setting the prefetch distance and fetching prefetched blocks to different storage levels at different times, the timeliness of prefetching is improved, cache pollution caused by prefetching is reduced, and the overall performance of the processor is improved.
[0083] Figure 4 shows a schematic diagram of the memory hierarchy proposed in this disclosure. Referring to Figure 4, the processor contains multiple levels of memory, including, for example, an on-chip N-level cache and off-chip memory. The cache typically consists of static random access memory (SRAM), including L1 cache, L2 cache, L3 cache, and so on up to the N-level cache (LN cache, also called the Last Level Cache, LLC). Off-chip memory typically consists of dynamic random access memory (DRAM). From L1 cache, L2 cache, L3 cache to the Last Level Cache LLC and off-chip memory, each memory level is further away from the processor core, with increasing capacity and access latency.
[0084] As shown in Figure 5, Figure 5 is a multi-level timely prefetch architecture diagram according to the present disclosure. Referring to Figure 5, the present disclosure provides a multi-level timely prefetch method and apparatus, including: a prefetching unit, a processor core, an on-chip N-level cache, and off-chip memory.
[0085] To address the issue of late prefetching and untimely prefetching, this disclosure sets a prefetch distance, skipping a portion of the header prefetch pattern before prefetching. This disclosure determines that a prefetch is too late when a normal memory access request accesses the cache, the prefetch request has already been sent, but the data has not yet returned; that is, the normal access request hits the prefetched block, but the data for the prefetched block has not yet been returned. In this case, by increasing the prefetch distance and sending a later pattern for prefetching, the untimely prefetching problem caused by late prefetching is resolved.
[0086] The training of the prefetch mode disclosed herein can be performed using access requests sent by the processor core to the L1 cache. Prefetch requests are sent to each level of cache, and the prefetching of each level of cache does not affect each other. The prefetch mode is sent to each level of cache for prefetching, and prefetching is performed separately in each level of cache, ultimately prefetching the L1 cache closest to the processor core. The processor's prefetching of the lower-level caches can achieve the goal of ensuring that the prefetch of the upper-level cache is hit in the lower-level cache, thus saving prefetching time. Each level of prefetch request is a normal access request for the lower-level cache. If the prefetch is too late, the prefetch distance is dynamically adjusted, thereby achieving timely prefetching of each level of cache.
[0087] The prefetch pattern generation disclosed herein is performed entirely in the Level 1 cache. The prefetch pattern generation is trained using access requests issued by the processor core; therefore, the prefetch pattern generation component resides only in the Level 1 cache. The generated prefetch patterns are then sent to each level of cache for prefetching, with different prefetch distances to each cache level.
[0088] As shown in Figure 6, which is a diagram of the prefetching components of each level of cache according to the present disclosure, each level of cache includes its own prefetching component. The multi-level timely prefetching of the present disclosure includes a prefetch access mode receiving component, a prefetch filtering component, and a prefetch issuing component.
[0089] The prefetch access mode receiving unit is responsible for receiving the prefetch access modes sent by the prefetch generation unit of the L1 cache.
[0090] The prefetching unit sends a prefetch request to the next level of storage system based on the prefetch distance.
[0091] The prefetch filter unit records the prefetch requests that have been issued. When a new prefetch request is received, the prefetch filter unit is checked first. If the prefetch request has already been issued, the same prefetch request will not be issued again, thus saving memory access bandwidth.
[0092] When each cache level receives an access request from the previous cache level (including prefetch requests from the previous cache level), it checks its own prefetch filtering unit to determine if a prefetch request has been issued but the data has not yet been returned. If so, it indicates that the current cache level's prefetch is not timely and the prefetch interval needs to be adjusted. Subsequent prefetches will then skip a certain prefetch interval and issue the next prefetch pattern.
[0093] The prefetch pattern generation method disclosed herein is applicable to various prefetch strategies, with the commonly used Spatial Memory Streaming (SMS) strategy as an example of prefetch pattern generation.
[0094] Spatial Memory Streaming (SMS) was proposed and evaluated in the context of server and scientific applications. The SMS hardware prefetcher utilizes spatial correlation. Whenever a spatial region is first requested, SMS uses information about misses to look up previously observed patterns in a table. As long as the application is using the spatial region, SMS begins observing and logging accesses to that region. Whenever a spatial region is no longer used (i.e., when the response blocks for the spatial region begin to be cleared from the cache), the access information observed by SMS is stored in a Pattern History Table (PHT).
[0095] Information in the Pattern History Table (PHT) is in the format of <event, pattern>.<event,pattern> The event is stored in the form of a program counter (PC) + address, a program counter (PC) + offset, an address, and an offset, i.e., PC + Address / PC + Offset / Address / Offset, etc., which is a piece of information associated with an observed access pattern. That is, if the event occurs again in the future, the corresponding storage access pattern will be used. For example, SMS selects the program counter (PC) + the offset that triggered the access, i.e., the PC of the first instruction to access the space region plus the distance between the first requested cache block and the start of the space region, as the event associated with the access pattern. In this way, whenever PC + Offset reappears, the relevant access pattern history will be used to issue a prefetch request. The pattern is the access history that occurred in each space region. SMS encodes the access pattern as a bit vector. In this way, for each cache block in the space region, a bit is stored to indicate whether the block has recently used the space region. If the block has recently used the space region, the bit is marked as "1", and if the block has not recently used the space region, the bit is marked as "0". Therefore, whenever a mode is used, prefetch requests are only issued for blocks in the storage mode where the corresponding bit is "1".
[0096] Figure 7 is a schematic diagram of the hardware implementation of the spatial storage stream prefetching strategy proposed in this disclosure.
[0097] This disclosure sends prefetch request patterns for spatial regions to each level of cache, i.e., 0x24D8, ..., 0x24F7. When a prefetch request is sent to the next level of storage system and it is found that the data has not been returned after the prefetch request has been sent, the distance between the prefetch requests and the next level of storage system is adjusted. For example, for the level 2 cache, the request is no longer sent from 0x24D8 but from 0x24F7. The adjustment of the prefetch request issuance can be shifted by one pattern request at a time, or multiple pattern requests at a time, dynamically adjusted based on the timeliness evaluation of prefetching.
[0098] The timeliness of prefetching can be evaluated based on its timeliness. For example, if a prefetching delay occurs for the first time, the prefetch request can be shifted back by one pattern request. If a delay occurs again, the prefetch distance can be adjusted more aggressively, such as shifting the prefetch request back by two pattern requests. The prefetch distance can also be adjusted based on other performance metrics such as the cache hit rate.
[0099] In a multi-level cache, a larger prefetch distance can be set for caches farther from the processor. The prefetch unit skips a certain prefetch distance before sending the data to the next level of cache for prefetching; for example, skipping a larger prefetch distance to the level 2 cache. This way, when the level 1 cache performs a prefetch, it can hit the level 2 cache, thus speeding up the level 1 cache prefetch. The specific prefetch distance skipped before sending the data to the next level of cache can be trained. For example, initially, the prefetch of each level of cache could be shifted two requests backward, and then dynamically adjusted based on the evaluation of untimely prefetching or the hit rate of each level of cache.
[0100] Figure 8 is a schematic diagram of the structure of a data prefetching device according to an embodiment of the present disclosure.
[0101] As shown in Figure 8, the data prefetching device 80 is applied to the processor, and the device includes:
[0102] The determination module 801 is used to determine the target data address from the multiple initial data addresses contained in the prefetch mode according to the prefetch distance after the nth level memory structure of the processor receives the prefetch mode and prefetch distance of the first level memory structure. Here, n is an integer greater than or equal to 1, and the prefetch distance is used to indicate the number of initial data addresses of the nth level memory structure that are not prefetched.
[0103] The sending module 802 is used to send a prefetch request corresponding to each target data address from the nth level storage structure to the (n+1)th level storage structure.
[0104] In some embodiments, the sending module 802 is configured to:
[0105] Determine the first matching result between the prefetch request and the historical prefetch record table, wherein the historical prefetch record table is used to record the prefetch request that the nth level storage structure has sent to the (n+1)th level storage structure;
[0106] When the first matching result indicates that the nth level storage structure has not sent the prefetch request to the (n+1)th level storage structure, the nth level storage structure sends the prefetch request to the (n+1)th level storage structure.
[0107] In some embodiments, the determining module 801 is further configured to:
[0108] After receiving a first access request from the corresponding upper-level storage structure, the nth-level storage structure determines a second matching result between the first access request and the historical prefetch record table. The historical prefetch record table is used to record the prefetch requests that the nth-level storage structure has sent to the (n+1)th-level storage structure. The upper-level storage structure corresponding to the first-level storage structure is the processor core of the processor.
[0109] The data prefetching device 80 also includes:
[0110] The update module is configured to: update the prefetch distance when the nth level storage structure meets preset conditions, wherein the preset conditions include: the second matching result indicates that the nth level storage structure has sent a prefetch request with the same target data address as the first access request to the (n+1)th level storage structure, and there is no data corresponding to the first access request in the nth level storage structure.
[0111] In some embodiments, the determining module 801 is configured to:
[0112] The target data address is determined from the plurality of initial data addresses contained in the prefetch pattern based on the updated prefetch distance.
[0113] In some embodiments, the update module is configured to:
[0114] Determine the prefetch performance metrics of the nth level memory structure;
[0115] The prefetch distance is updated based on the prefetch performance metrics.
[0116] In some embodiments, the prefetch performance metrics include at least one of the following:
[0117] The prefetch hit rate of the nth level storage structure;
[0118] The number of times the nth level storage structure continuously satisfies the preset conditions.
[0119] In some embodiments, the data prefetching device 80 further includes:
[0120] The generation module is configured to: receive a second access request from the processor core in the first-level memory structure of the processor, and generate the prefetch mode according to the second access request;
[0121] The determining module 801 is further configured to: determine the prefetch distance corresponding to the nth level storage structure, and send the prefetch mode and the prefetch distance to the nth level storage structure.
[0122] In some embodiments, the generation module is configured to:
[0123] Determine the target event identifier corresponding to the second access request;
[0124] The historical event identifier that matches the target event identifier is determined from the pattern history table, wherein the pattern history table includes a plurality of the historical event identifiers and a historical pattern corresponding to each historical event identifier;
[0125] The historical pattern corresponding to the matching historical event identifier is used as the prefetch pattern.
[0126] In some embodiments, the determining module 801 is configured to:
[0127] Determine the number of storage structure layers that separate the first-level storage structure from the nth-level storage structure;
[0128] Determine the initial distance corresponding to the nth level of the storage structure based on the number of storage structure layers;
[0129] Based on the initial distance, the prefetch distance corresponding to the nth level storage structure is determined.
[0130] In some embodiments, the determining module 801 is configured to:
[0131] The product of a preset distance value and the number of storage structure layers is determined as the initial distance.
[0132] In some embodiments, the determining module 801 is configured to:
[0133] Determine the prefetch performance index of the nth level storage structure during multiple data prefetching processes;
[0134] The initial distance is adjusted based on multiple prefetch performance metrics to obtain the prefetch distance corresponding to the nth level storage structure.
[0135] It should be noted that the foregoing explanation of the data prefetching method also applies to the data prefetching apparatus of the embodiments of this disclosure, and will not be repeated here.
[0136] In the embodiments of this disclosure, after receiving the prefetch mode and prefetch distance from the first-level storage structure, the nth-level storage structure of the processor determines the target data address from a plurality of initial data addresses included in the prefetch mode based on the prefetch distance, where n is an integer greater than or equal to 1, and the prefetch distance is used to indicate the number of initial data addresses for which the nth-level storage structure does not perform data prefetching; the nth-level storage structure sends a prefetch request corresponding to each target data address to the (n+1)th-level storage structure. This effectively improves the timeliness of prefetching, reduces cache pollution caused by prefetching, and thus improves the overall performance of the processor.
[0137] Figure 9 illustrates a block diagram of an exemplary computer device suitable for implementing embodiments of the present disclosure. The computer device 12 shown in Figure 9 is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present disclosure.
[0138] As shown in Figure 9, the computer device 12 is presented in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and a bus 18 connecting different system components (including system memory 28 and processing unit 16).
[0139] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MCA) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.
[0140] Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including volatile and non-volatile media, removable and non-removable media.
[0141] Memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and / or cache memory 32. Computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (not shown in Figure 9, commonly referred to as a "hard disk drive").
[0142] Although not shown in Figure 9, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disc drive for reading and writing to a removable non-volatile optical disc (e.g., a compact disc read-only memory (CD-ROM), a digital video disc read-only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this disclosure.
[0143] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of this disclosure.
[0144] Computer device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable human interaction with the computer device 12, and / or with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, computer device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0145] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the data prefetching method mentioned in the foregoing embodiments.
[0146] To implement the above embodiments, this disclosure also proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the data prefetching method proposed in the foregoing embodiments of this disclosure.
[0147] To implement the above embodiments, this disclosure also proposes a computer program product that, when the instruction processor in the computer program product is executed, performs the data prefetching method as proposed in the foregoing embodiments of this disclosure.
[0148] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.
[0149] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.
[0150] It should be noted that in the description of this disclosure, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this disclosure, unless otherwise stated, "a plurality of" means two or more.
[0151] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.
[0152] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0153] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
[0154] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.
[0155] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.
[0156] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0157] Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present disclosure.
[0158] All embodiments disclosed herein can be executed individually or in combination with other embodiments, and are all considered to be within the scope of protection claimed by this disclosure.
Claims
1. A data prefetching method applied to a processor, the method comprising: After receiving the prefetch mode and prefetch distance from the first-level storage structure, the nth-level storage structure of the processor determines the target data address from the multiple initial data addresses included in the prefetch mode according to the prefetch distance, where n is an integer greater than or equal to 1, and the prefetch distance is used to indicate the number of initial data addresses for which the nth-level storage structure does not perform data prefetching. The nth level storage structure sends a prefetch request corresponding to each target data address to the (n+1)th level storage structure.
2. The method as described in claim 1, wherein, Sending a prefetch request corresponding to each target data address to the (n+1)th level storage structure includes: Determine the first matching result between the prefetch request and the historical prefetch record table, wherein the historical prefetch record table is used to record the prefetch request that the nth level storage structure has sent to the (n+1)th level storage structure; When the first matching result indicates that the nth level storage structure has not sent the prefetch request to the (n+1)th level storage structure, the nth level storage structure sends the prefetch request to the (n+1)th level storage structure.
3. The method as described in claim 1 or 2, wherein, The method further includes: After receiving a first access request from the corresponding upper-level storage structure, the nth-level storage structure determines a second matching result between the first access request and the historical prefetch record table. The historical prefetch record table is used to record the prefetch requests that the nth-level storage structure has sent to the (n+1)th-level storage structure. The upper-level storage structure corresponding to the first-level storage structure is the processor core of the processor. When the nth level storage structure meets preset conditions, it updates the prefetch distance, wherein the preset conditions include: the second matching result indicates that the nth level storage structure has sent a prefetch request with the same target data address as the first access request to the (n+1)th level storage structure, and there is no data in the nth level storage structure corresponding to the first access request.
4. The method of claim 3, wherein, The step of determining the target data address from multiple initial data addresses included in the prefetch pattern based on the prefetch distance includes: The target data address is determined from the plurality of initial data addresses contained in the prefetch pattern based on the updated prefetch distance.
5. The method as described in claim 3 or 4, wherein, The updating of the prefetch distance includes: Determine the prefetch performance metrics of the nth level memory structure; The prefetch distance is updated based on the prefetch performance metrics.
6. The method of claim 5, wherein, The prefetch performance metrics include at least one of the following: The prefetch hit rate of the nth level storage structure; The number of times the nth level storage structure continuously satisfies the preset conditions.
7. The method of claim 6, wherein, The method further includes: The processor's first-level memory structure receives a second access request from the processor core and generates the prefetch mode based on the second access request; The first-level storage structure determines the prefetch distance corresponding to the nth-level storage structure and sends the prefetch mode and the prefetch distance to the nth-level storage structure.
8. The method of claim 7, wherein, The step of generating the prefetch pattern based on the second access request includes: Determine the target event identifier corresponding to the second access request; The historical event identifier that matches the target event identifier is determined from the pattern history table, wherein the pattern history table includes a plurality of the historical event identifiers and a historical pattern corresponding to each historical event identifier; The historical pattern corresponding to the matching historical event identifier is used as the prefetch pattern.
9. The method of claim 7 or 8, wherein, Determining the prefetch distance corresponding to the nth level storage structure includes: Determine the number of storage structure layers that separate the first-level storage structure from the nth-level storage structure; Determine the initial distance corresponding to the nth level of the storage structure based on the number of storage structure layers; Based on the initial distance, the prefetch distance corresponding to the nth level storage structure is determined.
10. The method of claim 9, wherein, The step of determining the initial distance corresponding to the nth level storage structure based on the number of storage structure layers includes: The product of a preset distance value and the number of storage structure layers is determined as the initial distance.
11. The method of claim 9 or 10, wherein, The step of determining the prefetch distance corresponding to the nth level storage structure based on the initial distance includes: Determine the prefetch performance index of the nth level storage structure during multiple data prefetching processes; The initial distance is adjusted based on multiple prefetch performance metrics to obtain the prefetch distance corresponding to the nth level storage structure.
12. A data prefetching apparatus applied to a processor, the apparatus comprising: A determining module is used to determine a target data address from a plurality of initial data addresses contained in the prefetch mode according to the prefetch distance after the nth level memory structure of the processor receives the prefetch mode and prefetch distance of the first level memory structure, wherein n is an integer greater than or equal to 1, and the prefetch distance is used to indicate the number of initial data addresses for which the nth level memory structure does not perform data prefetching. The sending module is used to send a prefetch request corresponding to each target data address from the nth level storage structure to the (n+1)th level storage structure.
13. A computer device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.
14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-11.
15. A computer program product, wherein, when instructions in the computer program product are executed by a processor, the method of any one of claims 1-11 is performed.