Data prefetching method, processor, electronic device, and computer device

Abstract

The application provides a data prefetching method, a processor, an electronic device and a computer device. The method is applied to a processor and includes the following steps: when a data prefetching operation is triggered by a memory access instruction, determining a cache hit rate; when the cache hit rate is lower than a set threshold, searching a matched entry from a pattern history table according to first index information and second index information; or when the cache hit rate is not lower than the set threshold, searching the matched entry from the pattern history table according to the first index information; wherein the searching accuracy of searching the matched entry from the pattern history table according to the first index information is higher than the searching accuracy of searching the matched entry from the pattern history table according to the second index information; determining a data prefetching address based on the searched entry, and reading data from the memory based on the data prefetching address. The above scheme can dynamically meet the data prefetching demand of the processor in different working states, and improve the data prefetching performance of the processor.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data prefetching method, processor, electronic device, and computer device. Background Technology

[0002] Prefetching techniques predict data that the processor will access and load it into the cache in advance, thereby reducing memory access latency and improving overall system performance. Footprint prefetching is a commonly used technique. Footprint prefetching is a footprint-based prefetching method. It divides memory into fixed-size footprint regions and records the memory blocks accessed within each region, representing the access pattern of that region as a bit vector (i.e., footprint). When an adjacent or new region is accessed, the footprint prefetcher predicts the memory blocks that may be accessed based on the previously recorded footprints and loads this data in advance, reducing cache misses and memory access latency.

[0003] During footprint prefetching, the processor's footprint prefetcher retrieves matching historical access patterns from a pre-built pattern history table based on information such as the instruction PC (Program Counter), access address offset, and address of the current memory access instruction, and determines the data prefetch address based on the retrieval results. Because the processor's load and memory access requirements vary at different times, the aforementioned footprint prefetching method is difficult to adapt to changes in the processor's data prefetching needs under different states. Summary of the Invention

[0004] To address the aforementioned technical problems, this application proposes a data prefetching method, processor, electronic device, and computer device, which enables the range and accuracy of the processor's data prefetching to be dynamically matched with the processor's operating state. This can dynamically meet the data prefetching needs of the processor under different operating states and avoid the impact of a single, fixed data prefetching mode on the processor's data prefetching performance under dynamically changing operating states.

[0005] The first aspect of this application proposes a data prefetching method applied to a processor, the method comprising:

[0006] Determine the cache hit rate when a memory access instruction triggers a data prefetch operation;

[0007] If the cache hit rate is lower than a set threshold, retrieve entries matching the first index information and entries matching the second index information from the pattern history table according to the first index information and the second index information; or, if the cache hit rate is not lower than the set threshold, retrieve entries matching the first index information from the pattern history table according to the first index information.

[0008] The entries in the pattern history table are used to record historical access records to memory regions; the first index information and the second index information include information about the memory access instructions; the retrieval accuracy of retrieving matching entries from the pattern history table based on the first index information is higher than the retrieval accuracy of retrieving matching entries from the pattern history table based on the second index information.

[0009] The data prefetch address is determined based on the retrieved table entries, and data is read from memory based on the data prefetch address.

[0010] In some implementations, the first index information includes the program counter value PC and the address of the memory access instruction;

[0011] The second index information includes the program counter value PC and the address offset of the memory access instruction.

[0012] In some implementations, when the cache hit rate is lower than a set threshold, determining the data prefetch address based on the retrieved table entries includes:

[0013] If the cache hit rate is lower than a first threshold, a first number of first entries and a second number of second entries are selected from the retrieved entries as target entries; or, if the cache hit rate is not lower than the first threshold and is lower than the set threshold, a third number of first entries and a fourth number of second entries are selected from the retrieved entries as target entries.

[0014] Wherein, the first threshold is less than the set threshold; the value of the second quantity divided by the first quantity is greater than the value of the fourth quantity divided by the third quantity; the first entry includes an entry that matches the first index information, and the second entry includes an entry that matches the second index information;

[0015] The data prefetch address is determined based on the target entry.

[0016] In some implementations, determining the data prefetch address based on the retrieved table entries includes:

[0017] Based on the virtual address of the memory access instruction, the target memory region accessed by the memory access instruction is determined from the retrieved table entries;

[0018] Based on the historical access records of the target memory region, the memory blocks in the target memory region that have been accessed in the past are identified as prefetch memory blocks;

[0019] The data prefetch address is determined based on the base address of the memory access instruction and the location of the prefetch memory block in the target memory region.

[0020] In some implementations, the retrieved table entries represent historical access records for each memory region using vectors;

[0021] Based on the virtual address of the memory access instruction, the target memory region accessed by the memory access instruction is determined from the retrieved table entries, including:

[0022] Based on the offset information in the virtual address of the memory access instruction, the target memory region accessed by the memory access instruction is located from the vector.

[0023] In some implementations, the pattern history table is constructed through the following processing:

[0024] During the processor's memory access process, the processor's memory access footprint is recorded in the form of a vector through a training table to obtain training table entries; one instruction corresponds to one training table entry, and the training table entry records the memory access footprint vector corresponding to that instruction.

[0025] When any entry in the training table is replaced, the replacement entry is stored in the pattern history table.

[0026] In some implementations, the method further includes, before storing the replacement item in the pattern history table:

[0027] Determine whether the number of accessed vector bits in the memory access footprint vector of the replacement item is greater than 1;

[0028] If the number of accessed vector bits in the memory access footprint vector of the replacement item is greater than 1, the replacement item is stored in the pattern history table.

[0029] In some implementations, the replacement item is stored in a pattern history table, including:

[0030] Using the instruction PC corresponding to the replacement item, retrieve the pattern history table entry corresponding to the instruction PC from the pattern history table;

[0031] If a pattern history entry corresponding to the instruction PC is retrieved from the pattern history table, the transformed memory access footprint vector is stored in the target sub-entry of the pattern history entry corresponding to the instruction PC, and the instruction address Address corresponding to the replacement item is stored as a label in the target sub-entry; wherein, the target sub-entry is the sub-entry corresponding to the address offset Offset of the replacement item in the pattern history entry corresponding to the instruction PC.

[0032] If no pattern history entry corresponding to the instruction PC is found in the pattern history table, a pattern history entry is assigned to the replacement item in the pattern history table, and the replacement item is written into the assigned pattern history entry.

[0033] In some implementations, storing the transformed memory access footprint vector into the target sub-entry of the pattern history table entry corresponding to the instruction PC includes:

[0034] If the target sub-entry in the mode history table entry corresponding to the instruction PC already stores the memory access footprint vector, the transformed memory access footprint vector is superimposed on the memory access footprint vector in the target sub-entry.

[0035] In some implementations, assigning a pattern history table entry to the replacement item in the pattern history table includes:

[0036] Determine whether there are any invalid pattern history entries in the pattern history table;

[0037] If an invalid pattern history entry exists, then the invalid pattern history entry is assigned to the replacement entry;

[0038] If no invalid pattern history entry exists, the least frequently accessed pattern history entry is assigned to the replacement entry.

[0039] A second aspect of this application provides a processor configured to perform the data prefetching method described above.

[0040] A third aspect of this application provides an electronic device, including a processor and a memory;

[0041] The memory is connected to the processor and is used to store programs;

[0042] The processor is used to implement the above-described data prefetching method by running a program in the memory.

[0043] The fourth aspect of this application provides a computer device including the processor described above.

[0044] The data prefetching method proposed in this application selects index information with different retrieval precisions from the pattern history table based on the processor's cache hit rate, and then performs data prefetching based on the retrieved pattern history table entries. This scheme enables the execution of pattern history table retrievals with different precisions using different index information when the processor is in different memory access states. This allows the range and precision of data prefetching to dynamically match the processor's operating state, dynamically meeting the processor's data prefetching needs under different operating states. It overcomes the inherent shortcomings of fuzzy prefetching and precise prefetching, and avoids the impact of a single, fixed data prefetching mode on the processor's data prefetching performance under dynamically changing operating states. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of a portion of the internal structure of a processor provided in an embodiment of this application.

[0047] Figure 2 This is a flowchart illustrating a data prefetching method provided in an embodiment of this application.

[0048] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0049] The technical solutions of this application are applicable to application scenarios where the processor prefetches data from memory, and are particularly suitable for processor footprint prefetching applications. By employing the technical solutions of this application, the processor can flexibly adjust its data prefetching method under different operating states, thereby enabling the data prefetching operation to adapt to changes in the processor's data prefetching requirements under different operating states and improving data prefetching efficiency.

[0050] With the development of computer architecture, processor speed and performance have continuously improved, but memory access speed remains relatively slow, becoming a bottleneck for system performance. To address this issue, hardware prefetching technology has been widely adopted. Prefetching technology predicts the data that the processor will access and loads it into the cache in advance, thereby reducing memory access latency and improving overall system performance.

[0051] Current hardware prefetching techniques are mainly divided into two categories: temporal prefetching and spatial prefetching.

[0052] Temporal locality-based prefetching leverages the temporal locality of data access, assuming that data accessed within a certain time period may be accessed again in the following time period. By preloading this data that is about to be accessed again, temporal locality-based prefetching reduces memory access latency.

[0053] Spatial locality-based prefetching is based on the spatial locality of data access. It assumes that after accessing a memory address, its adjacent addresses may also be accessed, and therefore, data blocks at these adjacent addresses are pre-loaded. Spatial locality-based prefetching assumes that memory access patterns are spatially repetitive. For example, if an access visits a series of addresses near X, then a similar sequence of addresses may be accessed near Y. This prefetching leverages the repetitiveness of data structures, also known as pattern locality.

[0054] In practical applications, spatial prefetching techniques are widely used, among which the most common are sequential prefetching, stride prefetching, and footprint prefetching.

[0055] Sequential prefetching: This assumes that memory access is performed sequentially, so adjacent memory blocks are prefetched.

[0056] Step prefetch: Assuming memory accesses are performed in fixed steps, the next memory block calculated in fixed steps is prefetched.

[0057] Footprint prefetching is a region-based prefetching method that divides memory into fixed-size footprint regions and records the accessed memory blocks within each region, representing the access pattern of that region as a bit vector (i.e., footprint). When accessing an adjacent or new region, the footprint prefetcher predicts the memory blocks that may be accessed based on the previously recorded footprints and preloads this data to reduce cache misses and memory access latency.

[0058] The Bingo prefetcher is a commonly used footprint prefetcher, and its workflow is as follows:

[0059] A) Memory access monitoring: When the processor performs memory access operations, the Bingo prefetcher monitors these memory accesses in real time and records the PC (program counter) and the memory address accessed for each access through a filter table (FT). Each time a new memory region is accessed, that region is stored in the FT.

[0060] B) Accumulation Table Records: If other unaccessed data blocks in this region are subsequently accessed, the Bingo prefetcher will transfer the region's information from the FT to the Accumulation Table (AT). The AT records the access information for this region and represents it in bit-vector form: 1 indicates that it has been accessed, and 0 indicates that it has not been accessed.

[0061] C) Training Complete: The training process will be completed when one of the following two events occurs: the entry corresponding to a certain region in AT is replaced, or the data block of that region is replaced from the cache. After training, the spatial access vector (i.e., footprint) will be associated with the triggering event of that spatial region and stored in the Pattern History Table (PHT) in the form of <triggering event, access footprint>.

[0062] D) Triggered Prefetching: When a specific condition is met (such as accessing a new Region or a cache miss), the Bingo prefetcher uses the PC and Offset of the current memory access instruction, or the PC and Address of the current memory access instruction, as an index to look for similar access patterns previously recorded in the PHT. If the PC and Address index is successful, it is considered an exact prefetch, and the prefetch address is generated using the indexed PHT entry; if the PC and Address index fails, but the PC and Offset index is successful, it is considered a fuzzy prefetch, and the prefetch address is generated using the indexed PHT entry.

[0063] Although fuzzy prefetching and precise prefetching each have their advantages in different scenarios, they both have significant shortcomings:

[0064] A) Shortcomings of fuzzy prefetching:

[0065] Cache pollution: Fuzzy prefetching tends to perform extensive prefetching under uncertain access patterns, which can easily lead to a large number of invalid prefetch requests, increase the cache burden, and thus pollute the cache, affecting system performance.

[0066] Bandwidth waste: Because fuzzy prefetching has a wide coverage but insufficient accuracy, it may prefetch a large amount of unnecessary data, consuming limited memory bandwidth and reducing system efficiency.

[0067] B) Shortcomings of precise prefetching:

[0068] Limited coverage: Prefetching relies on accurate recording of past access patterns and is suitable for scenarios with stable and regular patterns. However, it is difficult for precise prefetching to work effectively when memory access patterns are complex, variable, or irregular.

[0069] Poor flexibility: Precise prefetching is not adaptable to changing memory access patterns and cannot respond quickly to new access patterns, resulting in reduced prefetching efficiency.

[0070] This shows that the processor's load and memory access requirements vary at different times, and the existing footprint prefetching method is difficult to adapt to the changes in the processor's data prefetching requirements under different states.

[0071] To address the aforementioned technical problems, this application proposes a novel data prefetching scheme that enables the processor to flexibly adjust the data prefetching method under different operating states. This allows the data prefetching operation to adapt to changes in the processor's data prefetching requirements under different operating states, thereby improving data prefetching efficiency.

[0072] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0073] This application provides a data prefetching method that can be executed by a prefetcher in a processor.

[0074] Figure 1 A schematic diagram of part of the processor's internal structure is shown. See also... Figure 1 As can be seen, the prefetcher (PF) is located between the Load Store Unit (LSU) and the Memory Management Unit (MMU). The LSU can prefetch data from the MMU through the prefetcher (PF), or the LSU can directly send data requests to the MMU.

[0075] The MMU can include a cache, and the MMU can perform memory accesses. When the MMU receives a data request from the LSU or PF, it can retrieve the data from the cache and send it back to the LSU or PF (the PF then sends the data back to the LSU). Alternatively, the MMU can directly load data from memory into the cache or read data from memory and send it back to the PF or LSU.

[0076] The data prefetching method proposed in this application is mainly applied between the processor and the memory system to improve the processor's cache hit rate and reduce memory access latency. The data prefetching method proposed in this application can be applied to the following typical application scenarios:

[0077] High-performance computing: In applications that require a large amount of memory access, such as scientific computing, image processing, and big data analysis, the data prefetching method of this application embodiment can significantly improve the efficiency of memory access and reduce the performance bottleneck caused by cache misses during the computing process.

[0078] Embedded systems: In resource-constrained embedded systems, the data prefetching method of the embodiments of this application can optimize cache utilization, reduce system power consumption, and improve overall performance.

[0079] Servers and Data Centers: In high-load environments of servers and data centers, the data prefetching method of this application can reduce memory access latency and improve system response speed and throughput.

[0080] As the basis for prefetching memory data, in this embodiment of the application, the prefetcher monitors the processor's memory access process in real time and records and summarizes the memory access footprint to obtain a pattern history table.

[0081] During the processor's memory access process, the prefetcher records the processor's memory access footprint in the form of a vector using a training table, resulting in training table entries. Each memory region corresponds to one training table entry, which records the processor's memory access footprint vector for that memory region.

[0082] Specifically, when the prefetcher detects that the processor is accessing memory, the training table in the prefetcher will detect the current memory access pattern in real time, that is, detect the PC, Offset, and Address of the current memory access instruction, determine the 1K memory region being accessed and the specific memory block being accessed, and record it to obtain the memory access pattern corresponding to the 1K memory region.

[0083] Then, the prefetcher retrieves a matching entry from the training table using the address of the 1K memory region recorded in the memory access pattern. If the memory access pattern matches an entry in the training table, the footprint of the 1K space corresponding to the memory access pattern is marked in the entry and recorded in the form of a bit-vector. In this embodiment, the 1K memory space is divided into 16 64B cachelines, each cacheline corresponding to one bit in the bit-vector. If a bit in the bit-vector is 1, it indicates that the corresponding cacheline has been accessed; if a bit in the bit-vector is 0, it indicates that the corresponding cacheline has not been accessed.

[0084] If the memory access pattern does not match an entry in the training table, the accessed 1KB memory space and the specific memory block are determined based on the PC, Offset, and Address of the current memory access instruction. A new training entry is then allocated, with its PC and Offset being the same as the memory access pattern's PC and Offset, its Address being the base address of the 1KB memory space, and its memory access footprint vector being the vector of access footprints to the 1KB memory space recorded in the memory access pattern. In other words, for a 1KB memory space, the PC and Offset of the first instruction accessing that memory space serve as the PC and Offset of the training entry for that 1KB memory space. Subsequent updates to the access footprint of that 1KB memory space only update the memory access footprint of subsequent access instructions to the training entry's memory access footprint vector; the PC and Offset of the training entry are not updated.

[0085] When any item in the training table is replaced, the replacement item is stored in the Pattern History Table (PHT) for further summarization and training.

[0086] As can be seen from the training table and pattern history table mentioned above, a combination of PC+Offset corresponds to a memory access footprint vector bit-vector, which is used to record the memory access footprint of a certain memory space.

[0087] In other embodiments, before storing the replacement item in the Pattern History Table (PHT), the prefetcher determines whether to store it in the PHT for further training based on the information in the replacement item.

[0088] First, the prefetcher determines whether the number of accessed bits in the memory access footprint vector bit-vector of the replacement item is greater than 1, that is, whether the number of bits set to 1 in the bit-vector of the replacement item is greater than 1.

[0089] If the number of accessed vector bits in the memory access footprint vector of the replacement item is greater than 1, the replacement item is stored in the pattern history table. If the number of accessed vector bits in the memory access footprint vector of the replacement item is not greater than 1, the replacement item is discarded.

[0090] If the number of times the accessed region recorded by the replacement item is greater than 1, it indicates that the accessed region has a certain degree of spatial locality, and its access pattern is worth further training. Therefore, the replacement item is stored in the PHT. If the number of times the accessed region recorded by the replacement item is not greater than 1, it indicates that the accessed region does not have spatial locality and is not suitable for further training of the footprint prefetching strategy. Therefore, it can be discarded.

[0091] In other embodiments, it is also disclosed that the memory access footprint vector bit-vector in the replacement table is extended before the replacement table is stored in the PHT.

[0092] As described in the above embodiment, the memory access footprint vector bit-vector has a length of 16 bits, where each bit corresponds to a memory block in a memory region. This bit-vector is used to record the processor's access history of each memory block in the memory region.

[0093] To reduce storage overhead, this application embodiment expands the memory access footprint vector bit-vector and stores the stored memory access footprint vector bit-vector in PHT, thereby saving storage space.

[0094] Specifically, the prefetcher takes the position corresponding to the access address offset of the training entry in the memory access footprint vector stored in the training entry as the center position, and expands the number of vector bits on both sides of the center position of the memory access footprint vector to N-1 bits respectively, where N is the number of memory blocks included in the specific size memory region.

[0095] In this embodiment of the application, a 1K memory region is divided into 16 cache lines, so N is 16.

[0096] In other words, the prefetcher determines the value of the access address offset from the training entries, and then determines the position corresponding to the offset in the memory access footprint vector of the training entries, which is used as the center position.

[0097] Then, starting from this center position, the number of vector bits on both sides of the memory access footprint vector is expanded to 15 bits (high or low bits are supplemented with vector bits, and the value of the supplemented vector bits is set to 0), and the expanded 15-bit vector is concatenated. Furthermore, the value of each vector bit in the expanded memory access footprint vector is expanded to 2 bits.

[0098] For example, assuming the offset of the training entry is 9, the 9th vector position in the memory access footprint vector of the training entry is taken as the center position. Then, the number of vector bits to the left of the center position is expanded to 15 bits, and the number of vector bits to the right of the center position is expanded to 15 bits. The two expanded 15-bit vectors are concatenated, and the value of each vector bit is expanded to 2 bits.

[0099] After the above processing, the memory access footprint vectors corresponding to different offsets of the same PC can be superimposed and stored, without having to store the corresponding memory access footprint vector for each offset of the same PC separately, thus saving storage space.

[0100] Moreover, the above method of recording the memory access footprint corresponding to a certain offset records the access footprint of each memory block at a relative position relative to the offset, rather than the memory block access footprint at an absolute position. This ensures that the relative positions are consistent when storing memory block access footprints of different offsets, making it easier to perform the stacking operation.

[0101] Based on the bit-vector storage method for memory access footprint vectors described above, when the replacement item is stored in the PHT, the memory access footprint vector in the replacement item is first expanded into a 30-bit vector (each bit is 2 bits) according to the vector expansion method described above, resulting in the transformed memory access footprint vector. This transformed memory access footprint vector can then be stored in the PHT. For example, it can be superimposed on the memory access footprint vector corresponding to the same PC stored in the PHT, or a separate entry can be allocated for storing the transformed memory access footprint vector.

[0102] In other embodiments, before storing the transformed memory access footprint vector into the PHT, the existing entry corresponding to the instruction PC corresponding to the replacement item is retrieved from the PHT, that is, the pattern history entry with the same PC as the replacement item is retrieved from the PHT.

[0103] If an existing entry corresponding to the instruction PC is retrieved from the PHT, that is, a pattern history entry with the same PC as the replacement is retrieved, the transformed memory access footprint vector bit-vector of the replacement is stored in the target sub-entry of the existing entry, and the instruction address Address corresponding to the replacement is stored as a label in the target sub-entry; wherein, the target sub-entry is the sub-entry corresponding to the address offset Offset of the replacement in the existing entry.

[0104] In the embodiments of this application, one PC in the PHT corresponds to one pattern history entry. In the pattern history entry corresponding to each PC, there are a maximum of 16 sub-entries (one PC corresponds to a maximum of 16 offsets, and each sub-entry corresponds to one offset).

[0105] Using the PC of the aforementioned replacement item as the search condition, the pattern history table entry corresponding to the PC of the aforementioned replacement item is retrieved from the PHT. If the pattern history table entry corresponding to the PC of the aforementioned replacement item is found, the sub-item corresponding to the offset of the aforementioned replacement item is further retrieved from the pattern history table entry corresponding to the PC of the aforementioned replacement item. If the sub-item corresponding to the offset of the aforementioned replacement item is found, the transformed memory access footprint vector bit-vector of the replacement item is stored in the sub-item, and the instruction address Address corresponding to the replacement item is stored as a label in the sub-item.

[0106] If no sub-item corresponding to the offset of the above-mentioned replacement item is found in the pattern history table entry corresponding to the PC of the above-mentioned replacement item, then the sub-item corresponding to the offset of the above-mentioned replacement item can be added to the pattern history table entry corresponding to the PC of the above-mentioned replacement item. Then, the transformed memory access footprint vector bit-vector of the replacement item is stored in the added sub-item, and the instruction address Address corresponding to the replacement item is stored as a label in the added sub-item.

[0107] When storing the transformed memory access footprint vector (bit-vector) of the above replacement item into the above target sub-item, if the target sub-item already stores a memory access footprint vector, then the transformed memory access footprint vector of the above replacement item is superimposed on the memory access footprint vector already stored in the target sub-item (each vector bit consists of 2 bits, and increments by 1 for repeated access). If the above target sub-item is empty, then the transformed memory access footprint vector of the above replacement item is directly stored in the target sub-item.

[0108] If no existing entry for the instruction PC corresponding to the replacement item is found in the PHT, that is, no pattern history entry for the same PC as the replacement item is found, then a pattern history entry is assigned to the replacement item in the PHT, and the replacement item is written to the pattern history entry.

[0109] In some embodiments, when assigning a pattern history entry to the replacement item in the PHT, it is first determined whether there is an invalid pattern history entry in the PHT, that is, to search for an invalid pattern history entry in the PHT. If there is an invalid pattern history entry in the PHT, the first invalid pattern history entry found is assigned to the replacement item.

[0110] If there are no invalid schema history entries, the schema history entry with the lowest access frequency (i.e., the least recently used) is assigned to the replacement entry according to the Least Recently Used (LRU) algorithm.

[0111] After assigning a pattern history entry to a replacement item, the replacement item is written to that assigned pattern history entry. In this pattern history entry, the information of the replacement item is stored in the format of <trigger event, access footprint>, specifically...<PC|Offset|Address,Region> Wherein, Region is the memory access footprint vector corresponding to the triggering event represented by PC|Offset|Address.

[0112] Through the above processing, the prefetcher establishes and accumulates a series of footprint prefetch training data in the PHT. Based on this training data, data prefetching operations can be performed during subsequent processor memory accesses.

[0113] Based on the aforementioned pattern history table, this application proposes a data prefetching method, see [link to relevant documentation]. Figure 2 As shown, the method includes:

[0114] S101. When a memory access instruction triggers a data prefetch operation, determine the cache hit rate.

[0115] Specifically, the memory access instruction mentioned above refers to the processor's current memory access instruction, which contains the PC, Offset, and Address information of the instruction.

[0116] The current memory access instruction triggers a data prefetch operation, which may be because the memory access address of the current memory access instruction falls into a memory region that has not been accessed before. The system regards this as a new memory access pattern, which usually means that the processor is accessing a new data region and the previous access pattern may no longer be applicable. Therefore, the system needs to adjust and predict according to the new access pattern.

[0117] Alternatively, the current memory access instruction may trigger a data prefetch operation. This can also occur when the processor attempts to access data that is not in the cache using the current memory access instruction, i.e., when the current memory access instruction results in a cache miss. In this case, the system will trigger a prefetch operation to load data blocks that may be needed in the future.

[0118] When the processor's current memory access instruction triggers a data prefetch operation, the prefetcher calculates the processor's current cache hit rate, which is the probability that the processor will find the required data in the cache within a certain time period before the current moment. Assuming the processor requests data from the cache 100 times and finds the required data in the cache 60 times, the cache hit rate is 60%.

[0119] S102. If the cache hit rate is lower than a set threshold, retrieve entries matching the first index information and entries matching the second index information from the pattern history table according to the first index information and the second index information; or, if the cache hit rate is not lower than the set threshold, retrieve entries matching the first index information from the pattern history table according to the first index information.

[0120] The pattern history table mentioned above is the pattern history table constructed through the above embodiments. The entries in the pattern history table are used to record historical access records to memory regions.

[0121] The aforementioned first and second index information are composed of information from the memory access instructions, such as the PC, Offset, and Address of the memory access instruction. This first and second index information is used to retrieve pattern history entries from the aforementioned pattern history table that match the instruction information contained therein.

[0122] In this embodiment of the application, information based on memory access instructions is used to form first index information and second index information with different retrieval precision. The retrieval precision of retrieving matching entries from the pattern history table based on the first index information is higher than the retrieval precision of retrieving matching entries from the pattern history table based on the second index information.

[0123] In some embodiments, the first index information described above may consist of the PC+Address of the memory access instruction; the second index information described above may consist of the PC+Offset of the memory access instruction, or in other embodiments, the second index information described above may only include the PC of the memory access instruction.

[0124] When the processor's cache hit rate falls below a set threshold, the prefetcher retrieves entries from the pattern history table that match the first index and the second index, respectively. In this situation, because the processor's cache hit rate is low, the prefetcher needs to prefetch more data to improve it. Therefore, the prefetcher uses both the first and second indexes simultaneously for pattern history table retrieval. Since the first index has higher retrieval precision, it can retrieve pattern history entries that more precisely match the current memory access instruction. Conversely, since the second index has lower retrieval precision, it can retrieve more pattern history entries that match the current memory access instruction, thus improving the processor's cache hit rate.

[0125] When the processor's cache hit rate is not lower than a set threshold, the prefetcher retrieves a pattern history entry from the pattern history table that matches the first index information. In this case, because the processor's cache hit rate is high, it does not need to prefetch much data; it only needs to accurately prefetch the data that the processor might need. Therefore, by retrieving pattern history entries from the pattern history table using the first index information, the prefetcher can find pattern history entries that more precisely match the current memory access instruction, thereby reducing memory access bandwidth pressure and avoiding cache pollution, which is beneficial for improving processor performance.

[0126] S103. Determine the data prefetch address based on the retrieved table entry, and read data from memory based on the data prefetch address.

[0127] Specifically, after the prefetcher retrieves a pattern history entry from the pattern history table, it determines the data prefetch address based on the retrieved pattern history entry. This data prefetch address is the address of the memory block that has been accessed, as recorded in the retrieved pattern history entry.

[0128] For example, the prefetcher determines the location of a memory block that has been accessed in a memory space by retrieving table entries. Then, by adding the location of the accessed memory block to the base address of the memory space accessed by the current memory access instruction, it can obtain the data prefetch address of the memory space accessed by the current memory access instruction.

[0129] Then, the prefetcher reads data from memory according to the determined data prefetch address, that is, prefetches the data in the memory block of the memory space accessed by the current memory access instruction.

[0130] As described above, the data prefetching method proposed in this application selects index information with different retrieval precisions based on the processor's cache hit rate to retrieve pattern history entries from the pattern history table, and then performs data prefetching based on the retrieved pattern history table. This scheme enables the execution of pattern history table retrievals with different precisions using different index information when the processor is in different memory access states. This allows the scope and precision of data prefetching to dynamically match the processor's operating state, dynamically meeting the processor's data prefetching needs under different operating states. It overcomes the inherent shortcomings of fuzzy prefetching and precise prefetching, and avoids the impact of a single, fixed data prefetching mode on the processor's data prefetching performance under dynamically changing operating states.

[0131] In another embodiment, the ratio of retrieving pattern history entries from the pattern history table using the first index information and the second index information can be flexibly adjusted based on the processor's cache hit rate.

[0132] For example, in addition to setting a threshold for cache hit rate, a first threshold is also set, which is lower than the aforementioned threshold.

[0133] When the processor's cache hit rate is lower than the aforementioned first threshold, entries matching the first index information and entries matching the second index information are retrieved from the pattern history table based on the first index information and the second index information. From the retrieved entries, a first number of first entries and a second number of second entries are selected as target entries. The first entries are pattern history entries retrieved from the pattern history table using the first index information as the search condition; these are entries obtained through precise retrieval. The second entries are pattern history entries retrieved from the pattern history table using the second index information as the search condition; these are entries obtained through fuzzy retrieval.

[0134] If the processor's cache hit rate is not lower than the first threshold and is lower than the set threshold, the table entries that match the first index information and the table entries that match the second index information are retrieved from the pattern history table according to the first index information and the second index information. From the retrieved table entries, a third number of first table entries and a fourth number of second table entries are selected as target table entries.

[0135] Finally, the data prefetch address is determined based on the target entries obtained from the above retrieval.

[0136] Furthermore, in this embodiment, the value of the second quantity divided by the first quantity is greater than the value of the fourth quantity divided by the third quantity. That is, in the target entries used to determine the data prefetch address, the proportion of entries obtained by fuzzy retrieval increases as the processor cache hit rate decreases, and decreases as the processor cache hit rate increases. This allows for flexible adjustment of the ratio of fuzzy prefetching to precise prefetching according to changes in the processor's memory access working state, thereby improving the processor's memory access performance more dynamically and in a more refined manner.

[0137] In another embodiment, when the prefetcher determines the data prefetch address based on entries retrieved from the pattern history table, it does so through the following steps A1-A3:

[0138] A1. Based on the virtual address of the memory access instruction, determine the target memory region accessed by the memory access instruction from the retrieved table entries.

[0139] Specifically, based on bits [9:6] of the virtual address of the memory access instruction, the offset of the memory access instruction can be determined. Based on this offset, the memory region corresponding to the pattern history entry is determined as the target memory region. That is, the memory region recorded in the pattern history entry corresponding to the offset is the target memory region accessed by the memory access instruction.

[0140] In another embodiment, the vectors corresponding to the historical access records of each memory region corresponding to the same PC in the PHT are expanded and superimposed for storage, so that the historical access records of each memory region are represented by a 30-bit vector (each bit is 2 bits).

[0141] In this case, the target memory region accessed by the memory access instruction is located from the vector based on the offset information in the virtual address of the memory access instruction.

[0142] Specifically, after retrieving the corresponding pattern history entry from the PHT via the PC of the memory access instruction, the 30-bit memory access footprint vector is extracted from the pattern history entry.

[0143] Then, the offset of the memory access instruction is mapped to the 30-bit memory access footprint vector. For example, if the current offset is 9, the 9th bit of the 30-bit memory access footprint vector is used as the center, and 6 bits are taken to the left (6 bits to the left of the 9th bit is the boundary bit15 (a total of bit15-bit0, 16 bit-vectors)) and 9 bits are taken to the right to form a prefetch region, which is the target memory region.

[0144] In this embodiment, rules for extracting the prefetch region from the 30-bit memory access footprint vector are specified for different values ​​of Offset. Therefore, for any Offset, the range of the prefetch region can be extracted from the corresponding 30-bit memory access footprint vector according to the preset memory region selection rules for that Offset.

[0145] Furthermore, based on the memory region selection rules set for each offset, multiple prefetch regions can be determined from the 30-bit memory access footprint vector. Each prefetch region can serve as a target memory region. For example, for Offset = 0, three different prefetch regions can be formed by selecting vector bit ranges of 0-15, 9-24, and 16-30 from the 30-bit memory access footprint vector. These three prefetch regions can all serve as target memory regions for subsequent determination of data prefetch addresses. This allows data prefetching to be based on the access footprints of multiple adjacent memory regions, rather than being limited to the footprint of a single memory region.

[0146] A2. Based on the historical access records of the target memory region, the memory blocks that have been accessed in the target memory region are identified as prefetch memory blocks.

[0147] Specifically, from the pattern history entries corresponding to the target memory region, the memory access footprint vector bit-vector of the processor for the target memory region is determined. The memory blocks corresponding to the bits that are "1" in the vector are the memory blocks that have been accessed in the target memory region in the past, and these memory blocks are identified as prefetch memory blocks.

[0148] A3. Determine the data prefetch address based on the base address of the memory access instruction and the location of the prefetch memory block in the target memory region.

[0149] Specifically, bit[xx:10] of the virtual address of the memory access instruction is the base address of the memory access instruction. Using this base address, plus the location of the prefetch memory block in the target memory region, the data prefetch address can be calculated.

[0150] For example, assuming the vector bit values ​​corresponding to the 1st, 5th, 6th, and 7th memory blocks of the target memory region are 1, the data prefetch addresses are the four addresses: {va[xx:10] (base address), 4'b0001}, {va[xx:10], 4'b0101}, {va[xx:10], 4'b0110}, and {va[xx:10], 4'b0111}.

[0151] When there are multiple target memory regions determined through the above processing, for each target memory region, the corresponding data prefetch address can be determined based on the base address of the memory region accessed by the memory access instruction and the memory access footprint vector of the target memory region.

[0152] Corresponding to the data prefetching method described above, this application also proposes a processor configured to execute the data prefetching method described in any of the above embodiments. Specifically, the structure of this processor can be found in [reference needed]. Figure 1 As shown, when the processor executes the data prefetching method described in any of the above embodiments, it can be specifically executed by the prefetcher in the processor.

[0153] Based on the processor described above, embodiments of this application also propose a computer device that includes the processor described above.

[0154] For example, the computer device may specifically be a personal computer, server, work computer, workstation, smart terminal, smart wearable device, etc.

[0155] In the embodiments of this application, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a CPU, microprocessor, GPU, or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits, where the logical relationships of the hardware circuits are fixed or reconfigurable. For example, the processor is a hardware circuit implemented as an ASIC or PLD, such as an FPGA. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0156] Another embodiment of this application also provides an electronic device, see [link to relevant documentation] Figure 3 As shown, the device includes:

[0157] Memory 200 and processor 210;

[0158] The memory 200 is connected to the processor 210 and is used to store programs;

[0159] The processor 210 is configured to implement the data prefetching method disclosed in any of the above embodiments by running the program stored in the memory 200.

[0160] Specifically, the aforementioned electronic device may also include: a bus, a communication interface 220, an input device 230, and an output device 240.

[0161] The processor 210, memory 200, communication interface 220, input device 230, and output device 240 are interconnected via a bus. Among them:

[0162] A bus can include a pathway for transmitting information between various components of a computer system.

[0163] The processor 210 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0164] Processor 210 may include a main processor, as well as a baseband chip, modem, etc.

[0165] The memory 200 stores a program that executes the technical solution of this invention, and may also store an operating system and other key business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 200 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0166] Input device 230 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0167] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0168] The communication interface 220 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0169] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement the various steps of any of the data prefetching methods provided in the above embodiments of this application.

[0170] This application also proposes a chip including a processor and a data interface. The processor reads and runs a program stored in a memory through the data interface to execute the data prefetching method described in any of the above embodiments. For details of the processing and its beneficial effects, please refer to the embodiments of the above data prefetching method.

[0171] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps of the data prefetching method described in any of the above embodiments of this specification.

[0172] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0173] Furthermore, embodiments of this application may also be storage media storing a computer program, which is executed by a processor in the steps of the data prefetching method described in any of the above embodiments of this specification.

[0174] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0175] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0176] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0177] The modules and sub-modules in the various embodiments of the present application's devices and terminals can be merged, divided, and deleted according to actual needs.

[0178] It should be understood that the disclosed terminals, devices, and methods can be implemented in other ways, given the several embodiments provided in this application. For example, the terminal embodiments described above are merely illustrative. For instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0179] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0180] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0181] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0182] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0183] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0184] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data prefetching method, characterized in that, Applied to a processor, the method includes: Determine the cache hit rate when a memory access instruction triggers a data prefetch operation; If the cache hit rate is lower than a set threshold, retrieve entries matching the first index information and entries matching the second index information from the pattern history table according to the first index information and the second index information; and if the cache hit rate is not lower than the set threshold, retrieve entries matching the first index information from the pattern history table according to the first index information. The pattern history table stores training entries that have been replaced in the training table. Each training entry in the training table records the processor's memory access footprint in vector form, with one instruction corresponding to one training entry. Each training entry records the memory access footprint vector corresponding to that instruction. Each bit in the memory access footprint vector corresponds to a memory block in a memory region. The memory access footprint vector records the processor's historical access history to each memory block in the memory region. The first index information and the second index information include information about the memory access instructions. The retrieval accuracy of matching entries from the pattern history table based on the first index information is higher than the retrieval accuracy of matching entries from the pattern history table based on the second index information. The data prefetch address is determined based on the retrieved table entries, and data is read from memory based on the data prefetch address.

2. The method according to claim 1, characterized in that, The first index information includes the program counter value PC and the address of the memory access instruction; The second index information includes the program counter value PC and the address offset of the memory access instruction.

3. The method according to claim 1, characterized in that, When the cache hit rate is lower than a set threshold, determining the data prefetch address based on the retrieved table entries includes: If the cache hit rate is lower than a first threshold, a first number of first entries and a second number of second entries are selected from the retrieved entries as target entries; or, if the cache hit rate is not lower than the first threshold and is lower than the set threshold, a third number of first entries and a fourth number of second entries are selected from the retrieved entries as target entries. Wherein, the first threshold is less than the set threshold; the value of the second quantity divided by the first quantity is greater than the value of the fourth quantity divided by the third quantity; the first entry includes an entry that matches the first index information, and the second entry includes an entry that matches the second index information; The data prefetch address is determined based on the target entry.

4. The method according to claim 1, characterized in that, The process of determining the data prefetch address based on the retrieved table entries includes: Based on the virtual address of the memory access instruction, the target memory region accessed by the memory access instruction is determined from the retrieved table entries; Based on the historical access records of the target memory region, the memory blocks in the target memory region that have been accessed in the past are identified as prefetch memory blocks; The data prefetch address is determined based on the base address of the memory access instruction and the location of the prefetch memory block in the target memory region.

5. The method according to claim 4, characterized in that, The retrieved entries use vectors to represent historical access records for each memory region. Based on the virtual address of the memory access instruction, the target memory region accessed by the memory access instruction is determined from the retrieved table entries, including: Based on the offset information in the virtual address of the memory access instruction, the target memory region accessed by the memory access instruction is located from the vector.

6. The method according to claim 1, characterized in that, Before storing the replacement item in the pattern history table, the method further includes: Determine whether the number of accessed vector bits in the memory access footprint vector of the replacement item is greater than 1; If the number of accessed vector bits in the memory access footprint vector of the replacement item is greater than 1, the replacement item is stored in the pattern history table; wherein, the replacement item is the training table item that has been replaced out of the training table.

7. The method according to claim 1, characterized in that, Store the replacement items in the pattern history table, including: Using the instruction PC corresponding to the replacement item, retrieve the pattern history table entry corresponding to the instruction PC from the pattern history table; wherein, the replacement item is the training table entry that has been replaced out of the training table. If a pattern history entry corresponding to the instruction PC is retrieved from the pattern history table, the transformed memory access footprint vector of the replacement item is stored in the target sub-item of the pattern history entry corresponding to the instruction PC, and the instruction address Address corresponding to the replacement item is stored as a label in the target sub-item; wherein, the target sub-item is the sub-item corresponding to the address offset Offset of the replacement item in the pattern history entry corresponding to the instruction PC; If no pattern history entry corresponding to the instruction PC is found in the pattern history table, a pattern history entry is assigned to the replacement item in the pattern history table, and the replacement item is written into the assigned pattern history entry.

8. The method according to claim 7, characterized in that, The step of storing the transformed memory access footprint vector into the target sub-entry of the pattern history table entry corresponding to the instruction PC includes: If the target sub-entry in the mode history table entry corresponding to the instruction PC already stores the memory access footprint vector, the transformed memory access footprint vector is superimposed on the memory access footprint vector in the target sub-entry.

9. The method according to claim 7, characterized in that, Assigning a pattern history table entry to the replacement item in the pattern history table includes: Determine whether there are any invalid pattern history entries in the pattern history table; If an invalid pattern history entry exists, then the invalid pattern history entry is assigned to the replacement entry; If no invalid pattern history entry exists, the least frequently accessed pattern history entry is assigned to the replacement entry.

10. A processor, characterized in that, The processor is configured to perform the data prefetching method as described in any one of claims 1 to 9.

11. An electronic device, characterized in that, Including processor and memory; The memory is connected to the processor and is used to store programs; The processor is configured to implement the data prefetching method as described in any one of claims 1 to 9 by running a program in the memory.

12. A computer device, characterized in that, Includes the processor as described in claim 10.

Images