A scheduling method for an indirect access data prefetcher

Abstract

A scheduling method for an indirect access data prefetcher, comprising the following steps: S100: constructing a data prefetcher for coping with irregular access; S200: using the data prefetcher to realize the identification and address prediction of the indirect access mode based on the difference matching between the index data and the indirect access address, and dividing the indirect access into different range groups, and performing the prefetch scheduling according to the range groups. The method improves the timeliness of the prefetched data, improves the accuracy and coverage of the data prefetching, and has extremely low hardware resource overhead.

Description

Technical Field

[0001] This disclosure pertains to the field of processor and cache technology, and particularly relates to a scheduling method for dealing with indirect access data prefetchers. Background Technology

[0002] Modern CPU architectures typically combine the advantages of both the von Neumann and Harvard architectures to optimize performance and memory access efficiency. In the von Neumann architecture, program instructions and data are stored in the same memory space, accessed by the CPU via a unified address bus. The Harvard architecture, on the other hand, separates instruction and data storage, using independent buses to access instructions and data in parallel, reducing conflicts. Modern CPUs usually combine these two approaches, using separate L1 instruction caches (L1 ICache) and data caches (L1 DCache) to achieve efficient access to instructions and data while maintaining the memory management methods of the von Neumann architecture. To further improve performance, CPUs employ multi-level cache systems (such as L1, L2, and L3 caches). Cache hits reduce memory access latency, while cache misses manage parallel cache requests through the Cache Miss Status Register (MSHR), ensuring efficient data scheduling and processing. This architecture design, while ensuring smooth instruction and data processing, significantly optimizes access speed and response time, especially under high-frequency access demands, resulting in a significant improvement in overall system execution efficiency.

[0003] In modern CPUs, instruction execution is typically optimized using multi-level caching. First, when the CPU is ready to execute an instruction, it obtains the address of the current instruction from the program counter (PC) and searches for the instruction in the L1 instruction cache (L1 ICache). If the corresponding instruction is found in the L1 ICache (a cache hit), the CPU loads it directly into the instruction execution unit. If a cache miss occurs, the request continues to higher-level caches such as L2 and L3, and may even eventually reach main memory. In the event of a cache miss, the data is retrieved from main memory or a lower-level cache before being returned to the CPU for execution.

[0004] Data prefetchers are key hardware components in modern CPUs, used to predict data that a program might access in the future and preload it into the cache, thereby reducing latency caused by cache misses. They analyze cache access behavior, especially in cache miss cases, to obtain the address information of requests and generate access requests for future data based on prediction algorithms (such as linear prediction or historical access pattern analysis). These predicted addresses are added to a prefetch queue, and after deduplication and timeliness checks, the requests are sent to memory or a lower-level cache. In the event of a cache miss, this prefetched data is stored in the cache's free space. When the program actually needs this data, it is already in the cache and can be directly returned to the CPU as a cache hit, thus avoiding the high latency of accessing memory. Data prefetchers effectively improve system performance, reduce CPU idle time while waiting for data, and optimize cache utilization. However, prefetching mechanisms also have challenges, such as over-prefetching potentially wasting cache resources, or incorrect predictions potentially storing useless data in the cache, affecting access to other important data. Therefore, the design of the data prefetcher needs to balance prediction accuracy with the rational use of cache space in order to maximize the system's execution efficiency.

[0005] CSR (Compressed Sparse Row) is an efficient storage format for sparse matrices, particularly suitable for matrices where most elements are zero. In CSR, the non-zero elements of the matrix are compressed and stored in three arrays: the `val` array stores the non-zero elements; the `colidx` array stores the column index of each non-zero element; and the `rowptr` array stores the position of the first non-zero element in each row within the value array. This significantly saves memory and speeds up access to sparse matrices. In sparse matrix-vector multiplication, the sparse matrix needs to be indexed into the vector elements for multiplication and addition. The `rowptr` array is used to obtain the non-zero elements of each row in the sparse matrix, and the `colidx` array is used to index the corresponding elements in the vector to perform the multiplication and addition operations and obtain the result.

[0006] Breadth-First Search (BFS) is a graph traversal algorithm that starts with a primary node, visits all its adjacent nodes, then visits the unvisited adjacent nodes of those adjacent nodes, expanding level by level until all reachable nodes have been traversed. BFS uses a queue to implement the level-by-level traversal, ensuring that nodes visited earlier are processed first. This algorithm is commonly used to solve problems related to shortest paths, graph connectivity, and level-order traversal. Because BFS expands level by level, it guarantees finding the shortest path from the starting point to the target node, making it particularly suitable for shortest path problems in unweighted graphs.

[0007] In the hierarchical memory architecture of modern CPUs, data requests between each cache level and the memory system (such as L1, L2, L3 caches and main memory) interact through dedicated data paths. To efficiently manage data flow and optimize system performance, certain functional units can monitor these data paths, reading and analyzing the information transmitted in real time. The purpose of monitoring is to filter out information crucial to system operation from the large volume of data and pass this information to other relevant functional units for further processing. For example, in a CPU cache system, the monitoring unit might monitor requests at each cache level, including cache hit and miss information, and adjust cache replacement, data prefetching, or cache coherence strategies based on this information. Furthermore, the monitoring unit can dynamically adjust memory access priorities based on program behavior, or trigger corresponding optimization operations (such as initiating data prefetching or adjusting cache strategies) when specific events occur. Monitoring is not limited to caches; it may also involve interactions with other hardware resources, such as bus monitoring and instruction scheduling. Monitoring functionality is particularly important in multi-core processors and complex memory hierarchy systems, helping the CPU better adapt to program behavior patterns, improve cache hit rates, reduce access latency, and thus improve overall computational efficiency. By effectively monitoring and managing the data path, the CPU can make smarter resource allocation and optimization decisions in dynamic and changing execution environments.

[0008] Indirect access is a common irregular memory access pattern, typically occurring in scenarios where data needs to be accessed based on dynamically calculated addresses. The basic form of indirect access is calculating the access address of another array (e.g., x) using an index array (e.g., a[i]), in the form of x[a[i]]. In this case, a[i] is not a fixed address but is dynamically generated by some calculations during program runtime, leading to discontinuous and irregular data access. Due to the irregularity of this access pattern, indirect access usually results in a low cache hit rate because the data access pattern does not match the physical layout in memory, making it difficult to utilize the spatial locality of the cache. Indirect memory access is prevalent in the aforementioned improved CSR format calculations.

[0009] Difference is a computational method primarily applied to a sequence, or in this case, a data stream. For a sequence or data stream, subtracting the previous term from each non-first term yields a new sequence or data stream, which can be called the difference between the original sequence or data stream. Difference matching involves comparing two difference sequences and identifying their identical components.

[0010] In modern graph computing and sparse matrix operations, irregular computation patterns are common. These memory access patterns involve irregular access to memory, meaning that the accessed data addresses exhibit a certain degree of randomness. Such access patterns pose challenges to cache structures and prefetching strategies, often resulting in high cache miss rates, which in turn increases memory access latency and severely impacts computational performance.

[0011] Currently, traditional data prefetchers often use a single prefetch triggering method to identify corresponding memory access patterns, and the lookahead and priority of prefetched data blocks are often fixed. This design makes it difficult to control the timeliness of prefetching. If the prefetched data enters the cache too early compared to the CPU request, the cache replacement mechanism may evict the prefetched data, resulting in a miss when the CPU finally requests the data. If the prefetched data is sent too late compared to the CPU request, the prefetch request may still be waiting for the response data when the CPU requests the data. The timeliness of prefetching directly affects the cache hit rate when the CPU normally issues a memory access. Summary of the Invention

[0012] To address the aforementioned issues, this disclosure provides a scheduling method for indirect access data prefetchers, comprising the following steps: S100: Construct a data prefetcher to handle irregular access; S200: The data prefetcher uses differential matching between index data and indirect access addresses to identify indirect access patterns and predict addresses, and divides indirect access into different range groups, and performs prefetching scheduling according to the range groups.

[0013] Furthermore, the present invention also discloses a scheduling device for dealing with indirect access data prefetchers, comprising: A device for constructing a data prefetcher to handle irregular access; An apparatus for identifying and predicting indirect access patterns based on differential matching between index data and indirect access addresses using the data prefetcher, and for dividing indirect access into different range groups and performing prefetch scheduling according to the range groups.

[0014] Furthermore, the present invention discloses a computer storage medium comprising computer instructions that, when executed on a computer, cause the computer to perform the method described thereon.

[0015] Furthermore, the present invention also discloses an electronic device, wherein the electronic device comprises: Memory, processor, and computer programs stored in memory and executable on the processor, wherein, The processor implements the method when executing the program.

[0016] Based on the above technical solutions, this method proposes a scheduling approach for indirect access data prefetchers. This data prefetcher (DMP, Differential Matching Prefetcher) addresses the data scheduling problem in out-of-order, irregular access. DMP, based on differential matching between index data and indirect access addresses, can capture indirect access patterns and predict addresses. Furthermore, it divides indirect access into different range groups and performs prefetch scheduling according to these range groups, thereby improving the timeliness, accuracy, and coverage of prefetched data. This DMP data prefetcher also boasts extremely low hardware resource overhead. Attached Figure Description

[0017] Figure 1 This is a flowchart of a scheduling method for dealing with indirect access data prefetchers provided in one embodiment of the present disclosure; Figure 2 This is a block diagram of the overall structure of a DMP provided in one embodiment of this disclosure; Figure 3 This is a schematic diagram of a differential matching module provided in one embodiment of the present disclosure; Figure 4 This is a prefetch queue list item provided in one embodiment of this disclosure; Figure 5 This is an indirect access mode table entry provided in one embodiment of this disclosure; Figure 6 This is a flowchart of the BFS algorithm provided in one embodiment of this disclosure; Figure 7 This is a schematic diagram of prefetch priority allocation and scheduling provided in one embodiment of this disclosure; Figure 8 This is a schematic diagram of prefetch look-ahead dynamic adjustment and prefetch address generation provided in one embodiment of this disclosure; Figure 9 This is a prefetch look-ahead dynamic adjustment state machine provided in one embodiment of the present disclosure; Figure 10 This is a schematic diagram of CSR prefetch restriction identification provided in one embodiment of this disclosure; Figure 11 This is a schematic diagram of a prefetch triggering method provided in one embodiment of this disclosure; Figure 12 This is a comparison chart of prefetch coverage of different sparse matrices provided in one embodiment of this disclosure; Figure 13 This is a comparison chart of the prefetching times of different sparse matrices provided in one embodiment of this disclosure; Figure 14This is a comparison chart of the prefetch accuracy of different sparse matrices provided in one embodiment of this disclosure. Detailed Implementation

[0018] To further describe the present invention, the following description is in conjunction with the accompanying drawings. Figures 1 to 14 Further explanation is needed.

[0019] In one embodiment, such as Figure 1 As shown, this disclosure provides a scheduling method for indirect access data prefetchers, which includes the following steps: S100: Construct a data prefetcher to handle irregular access; S200: The data prefetcher uses differential matching between index data and indirect access addresses to identify indirect access patterns and predict addresses, and divides indirect access into different range groups, and performs prefetching scheduling according to the range groups.

[0020] In this embodiment, the method can be mainly applied to chip architectures using multi-level storage structures to identify irregular indirect access patterns and schedule prefetching reasonably, thereby reducing cache resource overhead caused by premature prefetching, improving the timeliness of prefetching, and improving the operating efficiency of the computing architecture.

[0021] In another embodiment, the data prefetcher includes differential matching, a priority register to be allocated, a prefetch data block replacement register, an indirect access mode table, a prefetch address generation module, a prefetch range restriction module, a prefetch queue, and a prefetch status register.

[0022] In this embodiment, the DMP mainly consists of eight components, located in the first-level cache of the CPU memory access hierarchy. It also needs to be able to monitor access requests and data responses between the computing core and the first-level cache, as well as between the first and second-level caches. Specifically, as follows... Figure 2 As shown, the components include Differential Matching, Priority Allocation Register, Prefetch Replacement Register, Indirect Relation Table, AddressGenerating Module, Range Restriction Module, Prefetch Queue, and Prefetch Status Handler Register.

[0023] In another embodiment, the differential matching is responsible for identifying the indirect access mode, the priority register to be allocated and the prefetch data block replacement register are mainly responsible for two necessary pieces of information for prefetch request scheduling, the prefetch address generation module and the prefetch range restriction module constrain the prefetch request, and finally the prefetch queue and the prefetch status register are responsible for caching the prefetch request and maintaining the status after the prefetch request is issued.

[0024] In this embodiment, the differential matching module is the main logical operation unit for identifying indirect access patterns. This module performs differential operations on the data response returned from the L1 Cache to the CPU to obtain a differential sequence of index data. The index data is then compared in the comparison logic based on its starting position. Figure 3 As shown, the index data difference sequence is {8,7,10,16,18,5,5,4}. Every 3 items in this sequence are extracted as continuous subsequences for comparison. The first subsequence is {8,7,10} and the last subsequence is {5,5,4}. These subsequences will be compared in the comparison logic. The address information in the data request sent by the CPU core to the L1 cache is differentially processed to obtain the target address difference sequence. This target difference sequence is then compared using the comparison logic based on the different right-shifted bits. Figure 3 The target address difference sequence shown is {40, 64, 72}. This sequence is right-shifted by 0, 1, 2, and 3 bits respectively to obtain the sequences {40, 64, 72} to {5, 8, 9}. These sequences are then compared with the subsequences of the index data difference sequence in the comparison logic. If a match is found in the comparison logic, it indicates the existence of a corresponding indirect access prefetch relationship. This indirect prefetch access relationship is assigned a priority by the priority register and recorded in the indirect memory access mode table.

[0025] The priority register stores the current range group priority to be allocated. Each time a new range indirect access is detected, the priority in this register is assigned to the new range access, and the value of the register is incremented. The prefetch priority is divided into two parts: inter-group priority and intra-group priority. The inter-group priority increases as the indirect access mode chain deepens, while the intra-group indirect access priority is directly assigned to the intra-group priority on top of the existing inter-group priority and incremented.

[0026] The prefetch address generator calculates the final memory access address by looking up the base address, offset, and lookup in the indirect access pattern table, and based on data returned by the L2Cache. The base address, offset, and lookup are checked against the current streaming memory access program counter (PC) and index address in the table to ensure the accuracy of the calculated address. The formula for calculating the prefetch address is: Prefetch Addr = Base Addr + (Return data << Shift) + (Lookahead< <Shift) The generated data requests will be checked by the prefetch range limitation module and finally inserted into the prefetch queue. The prefetch queue will issue the prefetch requests according to the priority of different prefetch requests, and the status of the issued prefetch requests will be stored in the prefetch status register.

[0027] The prefetch range restriction module stipulates that after generating the prefetch address for each range indirect prefetch access mode, and after satisfying the identification format of CSR (i.e., there were multiple target PCs corresponding to the same index PC before, and the target PC is the index PC of this indirect access), the prefetch address of this prefetch must be within the upper and lower bounds.

[0028] The prefetch block replacement register is used to count the number of times a prefetched data block is replaced, which guides the increase and decrease of the prefetch look-ahead. When a prefetched data block in the L1 cache is replaced, it indicates that there is an error in the indirect access pattern recognition or that the prefetch look-ahead is too high. The indirect access pattern table needs to adjust the identified indirect access pattern and its prefetch look-ahead according to this indicator.

[0029] Each prefetch queue temporarily stores prefetch requests when the MSHR in the L1 cache is full. During the next scheduling cycle, the highest priority prefetch request will be prioritized for delivery. The corresponding entries in the prefetch queue are as follows: Figure 4 As shown, it is divided into a valid table identifier (Valid), which indicates that the prefetch request in this table is waiting to be issued; IRT ID identifies the position ID of the indirect relationship of the prefetch request in the indirect access mode table; and Priority indicates the priority of the current prefetch request, which is used when scheduling prefetch requests. When MSHR is not full, the prefetch request with the highest priority will be passed to the prefetch status processing register for final processing of the prefetch request.

[0030] The Prefetch Status Register (PSHR) is a storage unit used to store intermediate information during the generation of the target prefetch address from the index response data. Each entry is divided into four parts. The first part is a validity flag, indicating whether the current entry information is valid. The second part stores the ID of the corresponding mode in the Indirect Access Mode Table (IRT), used to index the corresponding entry information in the IRT. The third part is the prefetch address generated for the corresponding IRT entry and a specific indirect access mode; the access request generated by this address indicates that it is waiting for a response from the lower-level storage hierarchy. The fourth part is a sequence of 0s and 1s (Bitmap), used to identify which data in the lower-level response is useful (1 for useful, 0 for useless). The PSHR is mainly used to capture and process data returned from the lower-level hierarchy, and based on the identifier data in the Bitmap, it provides the address generator with further information to calculate the target prefetch address.

[0031] In another embodiment, the indirect access pattern table maintains a corresponding prefetch look-ahead for each indirect access pattern table entry, and the prefetch look-ahead is dynamically updated.

[0032] In this embodiment, the indirect access pattern table is a storage unit used to store the relationship between streaming memory access and indirect access. The table entries are divided into seven parts, such as... Figure 5As shown. The first part (Enable) stores flag information indicating whether prefetching is enabled for the indirect access mode corresponding to the table entry. The second and third parts store the index PC and target PC (i.e., the successfully matched Index PC and Target PC, which can be understood as corresponding to the instructions to load a[i] and loadx obtained from x[a[i]], respectively), provided by the differential matching module after a successful match. The fourth part (Base Addr) stores the base address corresponding to the indirect access. Among them, LastAddr is the most recently observed indirect access address in the target table. The fifth part (Shift) stores the offset bits used by the differential matching module for a successful match. The offset bits will be used for calculating the indirect access base address and generating the prefetch address. The sixth part (Indirection Type) indicates which of the following four types the indirect access mode belongs to: single-point non-redundant access, single-point redundant access, single-point redundant restricted access, and range access. Single-point non-redundant access is a normal single-point indirect access, generating a single-point prefetch request normally according to the designed prefetch triggering method. Single-point redundant access is a single-point indirect access identified as redundant and will not trigger prefetch generation. Single-point redundant restricted access is identified as access to CSR format data, generating a prefetch address, but it will not be issued as a prefetch request. Instead, the prefetch address is updated to the prefetch restriction value (boundary) of the current table entry's prefetch restriction target table entry as a prefetch restriction on the prefetch restriction target. Range access indicates that the access may require generating multiple prefetch requests to cover the complete memory address distribution range and needs to be verified by the prefetch range restriction module. Part VII, Priority, is the priority of each indirect access mode, reflected in the prefetch queue scheduling. If there are multiple prefetch schedules to be processed in the prefetch queue, the prefetch queue will first issue the higher-priority prefetch schedule to the prefetch status processing register. Other prefetch requests will be temporarily stored in the prefetch queue until the prefetch status processing register returns that the prefetch request has been processed. Lookahead is the lookahead parameter. In the initial straddle access and initial indirect access, the data address of the index in the index array is added with a fixed value of Lookahead to obtain the lookahead address for prefetching. Boundary is the upper bound of the indirect access mode index. That is, the address corresponding to the prefetch request triggered by this indirect access mode cannot exceed this upper bound; otherwise, the corresponding prefetch request will not be sent. LimitID indicates that the prefetch address generated by the corresponding indirect access mode should be constrained as the Boundary in the indirect access mode with ID LimitID, rather than being sent as a normal prefetch request.

[0033] In another embodiment, once the indirect access pattern is identified, a prefetch priority needs to be dynamically assigned, and the prefetch priority is determined by the prefetch priority to be assigned and the current range group.

[0034] In this embodiment, Figure 6 This demonstrates the code for Breadth-First Search (BFS) traversal of this method. The entire code includes streaming access to `bfs_queue` (i.e., accessing the offset address of the `bfs_queue` array by increasing it in fixed steps), single-point indirect prefetching of `row_ptr`, range indirect prefetching of `col_idx`, and single-point indirect prefetching of `result`. Figure 6 As shown, the instruction address for the memory access instruction to access bfs_queue is 0x400c70, and the instruction addresses for the memory access instructions to access row_ptr[i], row_ptr[i+1], col_idx, and result are 0x400c70, 0x400c7c, 0x400c80, 0x400c90, and 0x400ca0, respectively. The memory access data of bfs_queue will be used as the offset of row_ptr[i], that is, accessing row_ptr[3] = 3 and row_ptr[4] = 6. row_ptr[i] and row_ptr[i+1] will constrain the access range of col_idx, that is, the subsequence {2,7,8} of col_idx with a starting address offset of 3 and an ending address offset of 6 needs to be read. The memory access data {2,7,8} of col_idx will be used as the offset to read the result array to obtain the values ​​of result[2], result[7], and result[8].

[0035] Each indirect access pattern is defined by a fixed range group. A range group refers to the indirect access relationship group formed within the indirect access dependency chain, consisting of indirect access relationships where the access corresponding to the index PC value also exhibits cross-step access characteristics, and all indirect access relationships that use these indirect access relationships as the starting point of the chain until the next range indirect access. For example... Figure 7 As shown, the starting positions of the range groups are all address requests generated after range indirect access (such as loading colidx, which requires accessing a row of data in colidx) and streaming access (such as loading bfs_queue, which sequentially accesses the bfs_queue array). In BFS indirect access, bfs_queue is a streaming access, and col_idx is a range indirect prefetch. Therefore, the indirect access chain from bfs_queue to col_idx is range group 0, and the indirect access from col_idx to result belongs to range group 1.

[0036] After each indirect access pattern is identified, a prefetch priority needs to be dynamically assigned. The prefetch priority is determined by the prefetch priority to be assigned and the current range group. The prefetch priority is divided into intra-group priority and inter-group priority. In the fourth entry of the indirect prefetch table, the priority is 0b010001, where the high 3 bits 0b010 are the inter-group priority and the low 3 bits 0b001 are the intra-group priority. Each time a priority is assigned, the priority in the priority register to be assigned is incremented. If the next identified indirect access pattern belongs to a new range group, the inter-group priority is incremented by 1. If the next identified indirect access pattern belongs to a previous range group, the inter-group priority is inherited, and the intra-group priority is incremented by 1.

[0037] The allocation and scheduling of prefetch priorities can assign different prefetch priorities to the indirect access relationships in the indirect access dependency chain formed by multiple nested loops. Generally speaking, the number of deep loops is much higher than that of shallow loops, so deep access requests in the indirect access dependency chain are more frequent than shallow access requests. By allocating the priorities of different indirect access relationships according to range groups, the higher the prefetch priority can be achieved as the indirect access level is deeper, thus conforming to the order in which different indirect access relationships are identified in the indirect access dependency chain.

[0038] In another embodiment, indirect access mode scheduling -- prefetch look-ahead adjustment: the initial stride access of each range group is used as index data to prefetch the next level indirect access. The prefetch look-ahead refers to adding advance in the initial stride access or range indirect access of each range group. When executing the load col_idx instruction, the CPU needs to obtain the data in the address range from row_ptr[i] to row_ptr[i+1]. In the actual calculation of the prefetch address, the original calculation method is to add the base address to the target data right-shifted by the corresponding offset number. The calculation process is 0x60_3170 (base address) + target data << 2 (offset number). The base address and offset number are obtained through lookup indirection. The target data is the data returned to the CPU from L1 Cache when listening to the memory access of row_ptr[i]. The prefetch look-ahead will be increased on the original calculated prefetch address, and during the loop iteration, the prefetch address with the prefetch look-ahead will be sent to the prefetch pair. Finally, the calculation method will be modified to 0x60_3170 (base address) + target data << 2 (offset bits) + 3 (prefetch look-ahead) << 2 (offset bits).

[0039] The indirect access pattern table maintains a corresponding lookahead value for each entry. The lookahead value is dynamically updated in two ways: incrementing and resetting. Figure 8In this example, the default value for prefetch lookup is 2. This lookup level increases when the memory access address matches the memory access address generated by the prefetch request in MSHR. Figure 8 As shown, when the address of the data request issued by the CPU is found to be the same as the prefetch request address generated by entry 3, the prefetch lookahead maintained by the indirect access mode entry will be incremented from 2 to 3. When a data block replacement occurs in the L1 cache, the prefetch lookahead needs to be reset, such as... Figure 9 As shown, if the prefetched data block is replaced and the data block was generated by the prefetch request of table entry 3, then the lookahead of that table entry will be reset from 3 back to 2.

[0040] Figure 9 The lower half illustrates the lookahead update diagram used for prefetching. The prefetch lookahead increment is divided into two stages. Stage one uses the prefetch lookahead position of the current range group and the index data of the prefetch lookahead incremented by one to generate the prefetch address. Stage two then increments the prefetch lookahead value by one. This two-stage design ensures that no data is missed when the prefetch lookahead is incremented.

[0041] Since the actual time overhead of completing a prefetch request is not fixed and there may be certain dependencies between range groups, for example, if the access to the PC value 400c90 in range group 1 encounters a particularly long continuous address, a large number of CPU accesses will remain in range group 1. When the number of accesses exceeds a certain threshold, the CPU's access requests for data in range group 1 may replace the prefetched data in range group 0, thereby disrupting the ideal prefetch look-ahead balance of static look-ahead scheduling. In summary, the above-mentioned dynamic look-ahead mechanism reduces the impact of input data on prefetch scheduling, increases the robustness of the prefetch strategy and the timeliness of the first-level prefetch request.

[0042] In another embodiment, the indirect access mode scheduling-prefetch range constraint: When executing the load col_idx instruction, the CPU needs to fetch data in the address range from row_ptr[i] to row_ptr[i+1]. Therefore, the range and size of the prefetch look-ahead must be between row_ptr[i] and row_ptr[i+1]. This prefetch mode has a fixed characteristic: the addresses accessed by the two memory access instructions for row_ptr are both indirectly related to the loadbfs_queue instruction at address 0x400c70. When these two sets of indirect access relationships are identified, and the index PC values ​​are the same (i.e., the index PC in entries 0 and 1 is 0x400c70), and the next indirect access relationship in the indirect access chain is a range access (i.e., the index type in entry 2 corresponding to load col_idx is range), and this instruction depends on the load instruction at address 0x400c7c... The `row_ptr[i]` instruction provides the starting position, and the `loadrow_ptr[i+1]` instruction at address 0x40080 provides the ending position. This is reflected in the table entries as the index PC of entry 2 being the address of the `loadrow_ptr[i]` instruction. Therefore, entries 0 and 1 will constrain the prefetch range of entry 2. The limit ID of entries 0 and 1 in the indirect access relation table is 2, indicating that the prefetch data generated by entries 0 and 1 will constrain the prefetch address of entry 2.

[0043] The data response returned by the indirect prefetch pattern entry with LimitID is recorded in the range limiting module. Figure 10 The data responses for the prefetch requests corresponding to entries 0 and 1 are row_ptr[i] = 3 and row_ptr[i+1] = 6, respectively. When the entry corresponding to LimitID = 2 generates a prefetch address, the prefetch address is constrained by the upper and lower bounds formed by the previous prefetch values. That is, the offset of load col_idx plus the prefetch look-ahead should be within the range of 3 to 6. If the address is within the constraint range, the data prefetch request will be sent normally; otherwise, the data request will be abandoned.

[0044] Although adding prefetch lookup improves the timeliness of prefetching by prefetching indirect accesses in advance, the forward prefetching at the end of local contiguous access addresses is useless and will not be accessed by the CPU in the short term, thus increasing the possibility of prefetched data being replaced. Therefore, adding prefetch range limits to constrain the upper and lower bounds of prefetch lookup can effectively reduce the useless prefetching of the prefetch lookup mechanism after the end of local contiguous accesses, improve the available capacity of the cache and the effectiveness of DMP prefetching.

[0045] In another embodiment, the prefetching scheduling prefetching triggering method adopts a hybrid triggering method that combines access hit triggering and data backfilling triggering.

[0046] In this embodiment, there are two prefetch triggering methods: one is triggered when a normal memory access request from the CPU hits the L1 cache, generally called access hit triggering; the other is triggered when the L1 cache sends a request to the L2 cache after a miss, and the response data is filled, generally called data fill triggering. Access hit triggering is used for the initial indirect access pattern entry in each range group, meaning the prefetch is triggered after the CPU executes the corresponding index PC. Other indirect access pattern entries use data fill triggering, meaning the current prefetch operation is triggered after the data prefetched by the previous indirect prefetch entry in the range group arrives. For example... Figure 11 As shown, when the load bfs_queue instruction (PC value is 0x400c70) is executed, the CPU's access hit to the L1 Cache triggers a prefetch request for the indirect access mode table entry with index PC 0x400c70. This access hit triggers the starting indirect access mode table entry in range group 0. After calculating the prefetch request address and sending the prefetch request to the L2 Cache, when the L2 Cache backfills data to the L1 Cache, it triggers the prefetching of other entries in range group 0 that depend on entry 1. That is, because the index PC 0x400c7c in entry 3 is the same as the target PC in entry 1, the prefetch of entry 3 is triggered by the data backfill and then the prefetch request is sent to the L2 Cache for processing.

[0047] The triggering time of the access hit triggering method is strongly correlated with the CPU's normal access request. Therefore, the time interval between the prefetch request generated by this triggering method and the actual access request issued by the CPU is often short, thus ensuring the timeliness of prefetching. However, this triggering method requires the index data to exist in the L1 cache. This characteristic limits the prefetch coverage of the access hit event triggering prefetching. The data backfill triggering method can ensure that the prefetcher can definitely obtain the index data. However, since the actual data backfilling is often not synchronized with the actual PC execution of the CPU, the timing of prefetching may be earlier or later.

[0048] The hybrid triggering method combines the advantages of the two prefetching triggering methods mentioned above, enabling timely prefetching and effective control of prefetch coverage. The normal access triggering of the initial indirect access mode within the range group ensures synchronization with the current instruction stream of the CPU, while the data backfilling triggering of the other indirect access modes ensures that prefetching can continue along the indirect access dependency chain, avoiding interruption of the prefetching chain due to unavailable index data.

[0049] In another embodiment, the data prefetcher is located in the first-level cache of the CPU memory access hierarchy.

[0050] In another embodiment, the main metrics for measuring prefetcher performance are prefetch coverage, prefetch timeliness, and prefetch accuracy. Prefetch coverage refers to the percentage of cache misses reduced by prefetching out of the total number of cache misses without prefetching. Prefetch timeliness measures how many prefetch requests are actually accessed by the CPU and whose prefetched data has not been replaced by the cache. Prefetch timeliness assesses the percentage of timely prefetches out of all effective prefetches. Prefetch accuracy refers to the percentage of cache misses reduced by prefetching out of all issued prefetch requests. The calculation formula is as follows:

[0051]

[0052]

[0053] like Figure 12 Figure 13 Figure 14 These represent the prefetch coverage, prefetch timeliness, and prefetch accuracy of the indirect access prefetcher when processing different sparse matrices. Different parameter configurations are used when processing each set of matrices.

[0054] Breadth-First Search (BFS) and SpMV correspond to two sets of irregular memory access algorithms. BFS is a classic algorithm for graph traversal, which traverses the graph by visiting nodes level by level. In this embodiment, the graph data is stored in the CSR (Compressed Sparse Row) format of a sparse matrix. SpMV (Sparse Matrix-Vector Multiplication) is an important computational operation, the core of which is multiplying a sparse matrix with a dense vector. A sparse matrix is ​​a matrix with a low proportion of non-zero elements, while a dense vector is a vector whose elements are mostly non-zero.

[0055] This simulation experiment uses two sets of system parameters, A15 and N1, to simulate two commercial processors, Arm Cortex-A15 and Arm Neoverse N1, respectively, covering application scenarios in mobile embedded systems and cloud servers. In the DMP (Dynamic Prefetch) prefetcher configuration, the intra-group priority length is set to 8 bits, meaning each range group can accommodate 256 indirect access relationships. In the dynamic lookahead mechanism, the initial lookahead is set to 16 bytes under the A15 configuration and 24 bytes under the N1 configuration, with one prefetch replacement triggering a lookahead reset in both cases. Furthermore, the lookahead of the straddle prefetcher involved in the experiment is set to 4 cache blocks.

[0056] like Figure 12 As shown, the prefetch coverage and the prefetch percentage in the MSHR table are significantly correlated with the prefetch address translation failure rate. When a large number of prefetch addresses generated by the DMP are abandoned due to address translation failures, the number of actual prefetch requests will decrease accordingly, leading to a decrease in the prefetch percentage in the MSHR table and ultimately a decrease in prefetch coverage for indirect access. Specifically, in the Amazon-2008 and Web-Google test matrices, both the BFS prefetch coverage and the prefetch percentage in the MSHR table are below 50%, which significantly limits the effectiveness of the prefetch mechanism in improving the bottleneck of indirect access. However, from the overall test results, the DMP achieved an average prefetch coverage of 94% in the SpMV test and an average prefetch coverage of 70% in the BFS test.

[0057] like Figure 13 As shown, the prefetch timeliness rate reflects the proportion of prefetched data that is available when the CPU accesses it, out of all valid prefetch requests. Experimental results demonstrate that the Dynamic Prefetch Lookahead (DMP) mechanism achieves a relatively stable high timeliness rate across all test matrices. Specifically, DMP achieves an average prefetch timeliness rate of 95% for the SpMV algorithm and 92% for the BFS algorithm. This data fully demonstrates the significant advantage of DMP in terms of prefetched data availability, effectively ensuring that prefetched data is ready when the CPU accesses it, thereby improving overall system performance.

[0058] like Figure 14As shown, prefetch accuracy reflects the proportion of valid prefetch requests among the actual prefetch requests issued. Because DMP employs a range prefetch constraint mechanism, it effectively constrains the aggressiveness of range prefetching and significantly improves the proportion of valid prefetches by limiting the upper and lower bounds of the prefetch request addresses. Experimental data shows that DMP achieves average prefetch accuracy of 92% and 66% in SpMV and BFS tests, respectively. This result demonstrates that DMP can significantly improve prefetch performance in most test scenarios, effectively reduce invalid prefetch requests, and thus improve system resource utilization and overall performance.

[0059] In another embodiment, a scheduling apparatus for dealing with indirect access data prefetchers includes: A device for constructing a data prefetcher to handle irregular access; An apparatus for identifying and predicting indirect access patterns based on differential matching between index data and indirect access addresses using the data prefetcher, and for dividing indirect access into different range groups and performing prefetch scheduling according to the range groups.

[0060] Furthermore, the present invention discloses a computer storage medium comprising computer instructions that, when executed on a computer, cause the computer to perform any of the methods described above.

[0061] Furthermore, the present invention also discloses an electronic device, wherein the electronic device comprises: Memory, processor, and computer programs stored in memory and executable on the processor, wherein, When the processor executes the program, it implements any of the methods described above.

[0062] Although embodiments of the present invention have been described above in conjunction with the accompanying drawings, the present invention is not limited to the specific embodiments and application fields described above. The specific embodiments described above are merely illustrative and instructive, and not restrictive. Those skilled in the art can make many other forms based on the guidance of this specification and without departing from the scope of protection of the claims of the present invention, and all of these are within the scope of protection of the present invention.

Claims

1. A scheduling method for indirect access data prefetchers, comprising the following steps: S100: Construct a data prefetcher to handle irregular access; S200: The data prefetcher uses differential matching between index data and indirect access addresses to identify indirect access patterns and predict addresses, and divides indirect access into different range groups, and performs prefetching scheduling according to the range groups.

2. The method according to claim 1, preferably, the data prefetcher includes differential matching, a priority register to be allocated, a prefetch data block replacement register, an indirect access mode table, a prefetch address generation module, a prefetch range restriction module, a prefetch queue, and a prefetch status register.

3. According to the method of claim 2, the differential matching is responsible for identifying the indirect access mode, the priority register to be allocated and the prefetch data block replacement register are mainly responsible for two necessary pieces of information for prefetch request scheduling, the prefetch address generation module and the prefetch range restriction module will constrain the prefetch request, and finally the prefetch queue and the prefetch status register are responsible for caching the prefetch request and maintaining the status after the prefetch request is issued.

4. The method according to claim 2, wherein the indirect access pattern table maintains a corresponding prefetch look-ahead for each indirect access pattern table entry, and the prefetch look-ahead is dynamically updated.

5. According to the method of claim 1, after the indirect access mode is identified, a prefetch priority needs to be dynamically allocated, and the prefetch priority is determined by the prefetch priority to be allocated and the current range group.

6. The method according to claim 1, wherein the prefetching scheduling prefetching triggering method adopts a hybrid triggering method that combines access hit triggering and data backfilling triggering.

7. The method according to claim 1, wherein the data prefetcher is located in the first level cache in the CPU memory access hierarchy.

8. A scheduling apparatus for dealing with indirect access data prefetchers, comprising: A device for constructing a data prefetcher to handle irregular access; An apparatus for identifying and predicting indirect access patterns based on differential matching between index data and indirect access addresses using the data prefetcher, and for dividing indirect access into different range groups and performing prefetch scheduling according to the range groups.

9. A computer storage medium, wherein, The storage medium includes computer instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 7.

10. An electronic device, wherein, The electronic device includes: Memory, processor, and computer programs stored in memory and executable on the processor, wherein, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

Images