A cache access system supporting out-of-order processor data prefetching

Abstract

The application belongs to the technical field of integrated circuit design, and particularly relates to a cache access system supporting out-of-order processor data prefetching. The system specifically comprises a LOAD memory access information tracking and sequencing module, a LOAD memory access address history buffer, a prefetcher and a target prefetch address buffer. The LOAD memory access information tracking and sequencing module changes out-of-order LOAD memory access information into in-order LOAD memory access information, which is then input into the prefetcher; the prefetcher uses the in-order memory access information to realize more accurate training and target prefetch address prediction; valid target prefetch addresses output by the prefetcher are stored in the target prefetch address buffer to wait for subsequent sending; and the target prefetch address buffer is updated in real time to invalidate untimely addresses, so as to avoid sending useless prefetch addresses. The application can improve the learning efficiency of memory access rules and the accuracy of address prediction, and reduce the resource occupation of prefetch requests on the cache system.

Description

Technical Field

[0001] This invention belongs to the field of integrated circuit design technology, and specifically relates to a cache access system that supports out-of-order processor data prefetching. Background Technology

[0002] Out-of-order execution and out-of-order memory access are common techniques in modern out-of-order processors to improve instruction execution and memory access efficiency. By changing the order of instruction execution and memory access, the overall instruction execution time and memory access time are reduced. On the other hand, due to the complexity of applications and limitations such as cache size, cache miss rates are high for some memory-intensive applications. Memory access has gradually become a major performance bottleneck for modern processors. Data prefetching is an effective way to reduce cache miss rates and improve memory access efficiency. Advanced high-performance processors mostly use efficient data prefetchers to improve memory access performance. The prefetching technique that performs prefetching on data is called a data prefetcher. Data prefetching generally only prefetches LOAD data, which is the main bottleneck for memory access. Hardware-implemented data prefetchers mainly process the input memory access addresses and other memory access information according to the internally implemented prefetching algorithm, attempting to capture and reconstruct the memory access patterns of the application, that is, the patterns between memory access addresses. Then, they use the summarized memory access patterns to predict future memory access addresses, and finally "prefetch" the corresponding data at the predicted memory access addresses into the cache. If the prediction is accurate, the original cache miss is avoided. Since the data prefetcher needs to capture and reconstruct the memory access patterns of the application, it requires that the memory access addresses received by the data prefetcher are sequential. However, in out-of-order processors, memory access addresses are out of order. Using out-of-order memory access addresses will significantly affect the data prefetcher's ability to capture and reconstruct memory access patterns, thus affecting the accuracy of prediction. In addition, the prefetch requests issued by the data prefetcher require some resources of the caching system. Besides address prediction errors, accurate but untimely address prediction can also cause ineffective resource consumption and even affect the processing of normal memory access requests. Summary of the Invention

[0003] To overcome the shortcomings of existing technologies, this invention proposes a cache access system that supports out-of-order processor data prefetching and can filter out untimely prefetch requests.

[0004] This invention provides a cache access system supporting out-of-order processor data prefetching, primarily consisting of two parts: an address translation system that transforms out-of-order LOAD memory access information into sequential LOAD memory access information, and a invalidation system for the target prefetch address issued by the prefetcher. The specific hardware structure includes a LOAD memory access information tracking and sorting module, a LOAD memory access address history buffer, a prefetcher, and a target prefetch address buffer. The LOAD memory access information tracking and sorting module transforms out-of-order LOAD memory access information into sequential LOAD memory access information before inputting it into the prefetcher. The prefetcher can utilize sequential memory access information to achieve more accurate training and target prefetch address prediction. The valid target prefetch address output by the prefetcher is stored in the target prefetch address buffer, awaiting subsequent transmission. The target prefetch address buffer is updated in real time to invalidate outdated addresses to avoid sending useless prefetch addresses.

[0005] This invention utilizes the LOAD memory access sorting information in the load storage queue to sort out-of-order memory access addresses in the first-level data cache, thereby enabling more accurate training and prediction of the data prefetcher, and filtering out untimely prefetch requests through short-term memory access address history.

[0006] The cache access system for out-of-order processor data prefetching provided by this invention includes the following hardware components: a LOAD memory access information tracking and sorting module, a LOAD memory access address history buffer, a prefetcher, and a target prefetch address buffer; wherein:

[0007] The LOAD memory access information tracking and sorting module is a circular queue with three special position indices: the queue head, the queue tail, and the currently accessed entry located between the queue head and the queue tail. The queue head and queue tail correspond to the queue head and queue tail of the address queue loaded from the load memory unit. The number of entries in the circular queue is the same as the number of entries in the address queue loaded from the load memory unit. Each entry in the circular queue corresponds to an entry at the same position in the address queue. Since the address queue maintains sequential memory access information from the queue head to the queue tail, the circular queue also maintains sequential memory access information from the queue head to the queue tail. The contents of each entry in the circular queue include the LOAD address, the LOAD instruction PC, and address attributes such as whether it is cacheable, whether it can be fed forward, whether it is missing, and whether it is a hit. Among them, the LOAD address, LOAD instruction PC, address attributes such as whether it can be cached, and whether it can be fed forward can be directly obtained from the load address queue in the load memory unit. The missing and hit information needs to be obtained from the first-level data cache. When the memory access address is sent from the load memory unit to the first-level data cache, the missing and hit information obtained is written to the corresponding entry in the circular queue according to its position in the load address queue.

[0008] Furthermore, the conversion of out-of-order LOAD memory access information into sequential LOAD memory access information in the LOAD memory access information tracking and sorting module is specifically implemented as follows:

[0009] (1) The LOAD memory access information tracking and sorting module obtains the synchronous LOAD address, LOAD instruction PC, address attributes such as whether it can be cached and whether it can be fed forward from the load address queue in the load memory unit.

[0010] (2) After the LOAD address is sent to the first-level data cache to determine whether it is missing or hit, the missing and hit information is written to the corresponding entry in the circular queue of the LOAD memory access information tracking and sorting module according to the position of the LOAD address in the load address queue in the load memory unit.

[0011] (3) The circular queue within the LOAD memory access information tracking and sorting module has three special position indices: the queue head, the queue tail, and the current access entry located between the queue head and the queue tail. The queue head and queue tail are consistent with the queue head and queue tail of the address queue loaded from the load memory unit. Under normal circumstances, without exceptions or branch prediction errors, the queue head is submitted and updated when the LOAD instruction is submitted, and the queue tail is written and updated when the LOAD request is inserted. The current access entry located between the queue head and the queue tail stores the next address to be sent to the prefetcher, and the initial position of this entry is consistent with the queue head.

[0012] (4) In each cycle, the LOAD memory access information tracking and sorting module checks whether there is an address hit flag, an address missing flag, or an address feedforward flag in the current access entry. If at least any of the above three flags exists, the memory access information such as the LOAD address and LOAD instruction PC stored in the current access entry can be sent to the LOAD memory access address history buffer and prefetcher, and the position of the current access entry is updated at the same time.

[0013] The LOAD memory access address history buffer is a first-in-first-out (FIFO) queue. The number of entries in the queue is related to the time required for the prefetcher to generate the predicted target prefetch address from the input memory access information. The FIFO queue stores the most recent sequential memory access addresses input to the prefetcher by the LOAD memory access information tracking and sorting module. The memory access address history stored in the FIFO queue is used to determine whether the target prefetch address output by the prefetcher is valid, that is, whether the target prefetch address output by the prefetcher has been accessed by the processor within the prefetcher input-output delay time.

[0014] The prefetcher is a data prefetcher module. It has different implementations depending on the prefetching algorithm, but the input and output interfaces of the prefetcher are unified. The input is the sequential memory access information given by the LOAD memory access information tracking and sorting module, and the output is the predicted target prefetch address. The prefetcher performs pattern recognition and pattern capture on the input LOAD memory access information according to the specific prefetching algorithm, and then predicts the target prefetch address.

[0015] The target prefetch address buffer is a circular queue, where each entry stores a target prefetch address. After the target prefetch address generated by the prefetcher is determined to be valid, it is inserted into the tail of the circular queue. Each entry in the circular queue storing a valid target prefetch address may be invalidated externally before a corresponding prefetch request is issued. The target prefetch address buffer preferentially selects the first valid entry from the head of the queue to issue the corresponding prefetch request.

[0016] Furthermore, the failure system for the target prefetch address issued by the prefetcher in the target prefetch address buffer is specifically implemented as follows:

[0017] (1) When the prefetcher generates a new target prefetch address, it needs to determine whether the address is valid from two aspects. First, it needs to check whether there is the same address in the valid entries in the LOAD memory access address history buffer and the LOAD memory access information tracking and sorting module. If there is, it means that the target prefetch address has been actually accessed by the processor core recently, and the corresponding data is very likely to be in the cache, so the address is invalid. Second, it needs to check whether there is the same address in the valid entries in the target prefetch address buffer. If there is, it means that the target prefetch address has been sent recently and is a duplicate target prefetch address, so the address is invalid.

[0018] (2) After the target prefetch address generated by the prefetcher is determined to be a valid address, it will be inserted into the tail of the queue of the target prefetch address buffer. In each cycle, it will check whether the target prefetch address in each entry of the target prefetch address buffer is the same as the hit address or missing address given by the current level data cache. If the address stored in a certain entry is the same as that, it means that the address is being accessed and the address stored in the entry is no longer timely. The entry will be set as an invalid entry and will no longer be used for prefetching.

[0019] Compared with the prior art, the beneficial technical effects of the present invention are reflected in:

[0020] (1) The designed out-of-order memory access address to sequential memory access address conversion system utilizes the memory access address order provided by the load address queue in the processor's load memory unit to reorder the address and related information of the actual access to the first-level data cache and input them into the prefetcher, thereby realizing the pattern recognition and pattern capture of sequential memory access information by the prefetcher, improving the learning efficiency of memory access patterns and the accuracy of address prediction.

[0021] (2) The designed accurate but untimely target prefetch address invalidation system confirms whether the target prefetch address has been accessed by the processor before the corresponding prefetch request is actually issued by searching the short-term historical access address and the possible access address in the current period. Then, the found target prefetch address is invalidated, thereby avoiding the processing of such useless prefetch requests and reducing the resource occupation of the cache system by prefetch requests. Attached Figure Description

[0022] Figure 1 A block diagram of the hardware implementation of a cache access system to support out-of-order processor data prefetching.

[0023] Figure 2 The structural block diagram of the LOAD memory access information tracking and sorting module.

[0024] Figure 3 Block diagram of the target prefetch address buffer. Detailed Implementation

[0025] In this invention, the hardware implementation structure of the cache access system supporting out-of-order processor data prefetching is as follows: Figure 1 As shown, this cache access system is implemented at the L1 cache level, allowing easy communication with the processor's load memory. The specific hardware structure includes a memory access information tracking module, a memory access history buffer, a prefetcher, and a target prefetch address buffer. The hardware implementation does not involve the specific implementation of the prefetcher unit; instead, it processes the input and output of the prefetcher unit, using sequential memory access address inputs to assist the prefetcher in pattern recognition and rule capture, and filtering the target prefetch address output by the prefetcher to improve the prediction accuracy of the prefetcher and reduce the resource consumption of the cache system by prefetch requests.

[0026] All memory access requests issued by the processor core are processed by the load memory unit. The cache access system's memory access tracking module reads the LOAD memory access ordering information from the load memory unit and reads real-time out-of-order memory access information from the Level 1 data cache. The memory access information passed from the Level 1 data cache to the memory access tracking module is out of order; after reordering, the ordered memory access information is input into the prefetcher. The target prefetch address issued by the target prefetch address buffer is sent to the Level 2 data cache to read the corresponding data, and then the data is transferred to the Level 1 data cache. The functions of each module of the cache access system and its overall working principle are described below.

[0027] The structure of the memory access tracking module is as follows: Figure 2 As shown, this module is a circular queue responsible for reordering out-of-order memory access addresses given by the Level 1 data cache. Each entry in the circular queue stores six pieces of information: LOAD address, LOAD instruction PC, address attributes (cacheable, feedforwardable, missing, and hit). The LOAD address, LOAD instruction PC, address attributes (cacheable, feedforwardable), and missing / hit information can be directly obtained synchronously from the load memory unit's address queue. The missing and hit information needs to be obtained from the Level 1 data cache. When the memory access address is sent from the load memory unit to the Level 1 data cache, the missing / hit information is written to the corresponding entry in the circular queue according to its position in the load address queue. When the load memory unit receives a new LOAD memory access request from the processor core, the load address queue creates a new valid entry at the end of the queue to store the corresponding information. The circular queue of the memory access information tracking module is also updated synchronously, creating a new valid entry at the end of the queue to store the required corresponding information. When the processor submits a LOAD memory access instruction, it invalidates the entry at the head of the load address queue and also invalidates the entry at the head of the circular queue.

[0028] After a new LOAD memory access request enters the load memory unit, it will then be sent to the first-level data cache for data access. However, due to hardware structure conflicts, read-write conflicts, or other reasons, the memory access request sent to the first-level data cache may be rejected, thus requiring the load memory unit to send it again. It is precisely because some memory access requests are rejected that the memory access addresses processed by the first-level data cache are out of order, and the completion time of the memory access requests is out of order.

[0029] The circular queue within the memory access tracking module, like the load queue of the load memory unit, records the order of LOAD memory access requests from head to tail. A valid entry in the circular queue only indicates that the corresponding request is in the load memory unit, but it doesn't reveal whether the request has been sent to the first-level data cache or accepted. To mark this information for each entry, when a LOAD memory access request is sent to the first-level data cache and accepted, based on the current request's hit status and its position index in the load address queue, the hit or miss flag in the corresponding entry in the circular queue within the memory access tracking module is set to 1.

[0030] When the memory access tracking module outputs sequential memory access information, it searches for the first unsent entry that meets the conditions, starting from the head of the queue. The conditions for sending this entry are that any one of the following three flags—feedforward flag, hit flag, or missing flag—must be 1. Specifically, the feedforward flag refers to the feedforward from the write address detected in the loading memory unit to the read address. Since the read address is the same as a previous write address, the write request data can be directly fed forward to the read request data; the LOAD request with the feedforward flag does not actually need to be sent to the first-level data cache. The position of the currently accessed entry sent out is always between the head and tail of the queue, initially at the head, waiting to be sent out when the current accessed entry meets the aforementioned conditions. Therefore, the memory access tracking module's sending system ensures that the sent memory access information is sequential.

[0031] The memory access information sent by the memory access tracking module is input to the prefetcher and the memory address history buffer. The memory address history buffer is a first-in-first-out queue that stores access addresses within a recent period. Generally, the hardware implementation of the prefetcher is not purely combinational logic. There is a certain delay between the prefetcher reading the input memory access information and generating the corresponding prefetch address. During this delay, new memory access information may be output from the memory access tracking module. The main purpose of the memory address history buffer is to record the memory access addresses sent by the memory access tracking module during this delay. Since these sent memory access addresses in the memory access tracking module may be invalid, the addresses recorded in the memory address history buffer can be used to check whether the target prefetch address sent by the prefetcher has been accessed during the internal delay of the prefetcher, thereby determining whether the target prefetch address is outdated.

[0032] The prefetcher predicts the target prefetch address based on the sequential memory access information sent by the memory access information tracking module and after hardware processing by the internal prefetch algorithm.

[0033] The structure of the target prefetch address buffer is as follows Figure 3As shown, this buffer is a circular queue responsible for storing target prefetch addresses, expired target prefetch addresses, and outgoing prefetch requests. After a target prefetch address generated by the prefetcher is determined to be valid, it is inserted into the tail of the circular queue within the target prefetch address buffer. When the target prefetch address buffer sends a prefetch request, it preferentially selects the first valid entry from the head of the queue.

[0034] The validity of the target prefetch address generated by the prefetcher is determined mainly through two aspects: First, it is necessary to check whether there is a duplicate address in the valid entries of the LOAD memory access address history buffer and the LOAD memory access information tracking and sorting module. If there is, it means that this target prefetch address has been actually accessed by the processor core recently, and the corresponding data is very likely already in the cache, so the address is invalid. Second, it is necessary to check whether there is a duplicate address in the valid entries of the target prefetch address buffer. If there is, it means that this target prefetch address has been sent recently and is a duplicate target prefetch address, so the address is invalid.

[0035] Valid entries in the target prefetch address buffer may still expire before the corresponding prefetch request is sent. Each cycle, it is checked whether the target prefetch address in each entry is the same as the hit or missing address given by the current level-one data cache. If an entry stores the same address, it means that this address is being accessed, and the address stored in the entry is no longer timely. Therefore, the entry is set to invalid and will no longer be used for prefetching.

Claims

1. A cache access system supporting out-of-order processor data prefetching, characterized in that, The system is divided into two parts: an address translation system that converts out-of-order LOAD memory access information into sequential LOAD memory access information, and a failure system for the target prefetch address issued by the prefetcher. The hardware structure includes a LOAD memory access information tracking and sorting module, a LOAD memory access address history buffer, a prefetcher, and a target prefetch address buffer. The LOAD memory access information tracking and sorting module converts out-of-order LOAD memory access information into sequential LOAD memory access information before inputting it into the prefetcher. The prefetcher utilizes the sequential memory access information to achieve more accurate training and target prefetch address prediction. The valid target prefetch address output by the prefetcher is stored in the target prefetch address buffer, awaiting subsequent transmission. The target prefetch address buffer is updated in real time to fail outdated addresses to avoid sending useless prefetch addresses. Among these: The LOAD memory access information tracking and sorting module is a circular queue with three position indices: the queue head, the queue tail, and the currently accessed entry located between the queue head and the queue tail. The queue head and queue tail correspond to the queue head and queue tail of the address queue loaded from the load memory unit. The number of entries corresponds to the number of entries in the address queue loaded from the load memory unit, and each entry in the circular queue corresponds to the entry at the same position in the address queue loaded from the queue head. Since the address queue maintains sequential memory access information from the queue head to the queue tail, the circular queue also maintains sequential memory access information from the queue head to the queue tail. The contents of each entry in the circular queue include the LOAD address, LOAD instruction PC, address attributes (whether it is cacheable, whether it can be fed forward, whether it is missing, and whether it is a hit). Among them, the information of LOAD address, LOAD instruction PC, address attributes (whether it is cacheable, whether it can be fed forward) is directly obtained synchronously from the address queue in the load memory unit. The missing and hit information is obtained from the first-level data cache. When the memory access address is sent from the load memory unit to the first-level data cache, the missing and hit information obtained is written to the corresponding entry in the circular queue according to its position in the load address queue. The LOAD memory access address history buffer is a first-in-first-out queue. The number of entries in the queue is related to the time required for the prefetcher to generate the predicted target prefetch address from the input memory access information. The first-in-first-out queue stores the sequential memory access addresses most recently given to the prefetcher by the LOAD memory access information tracking and sorting module. The memory access address history stored in the first-in-first-out queue is used to determine whether the target prefetch address output by the prefetcher is valid. The prefetcher is a data prefetching module, which has different implementations depending on the prefetching algorithm. The input and output interfaces of the prefetcher are unified. The input is the sequential memory access information given by the LOAD memory access information tracking and sorting module, and the output is the predicted target prefetch address. The prefetcher performs pattern recognition and pattern capture on the input LOAD memory access information according to the specific prefetching algorithm, and then predicts the target prefetch address. The target prefetch address buffer is a circular queue, in which each entry stores a target prefetch address. The target prefetch address generated by the prefetcher is inserted into the tail of the circular queue after being determined to be a valid address. Each entry in the circular queue that stores a valid target prefetch address may be invalidated externally before the corresponding prefetch request is issued. The target prefetch address buffer preferentially selects the first valid entry starting from the head of the queue and then issues the corresponding prefetch request.

2. The cache access system supporting out-of-order processor data prefetching according to claim 1, characterized in that, In the LOAD memory access information tracking and sorting module, the process of converting out-of-order LOAD memory access information into sequential LOAD memory access information is as follows: (1) The LOAD memory access information tracking and sorting module obtains the synchronized LOAD address, LOAD instruction PC, address attributes (whether it can be cached, and whether it can be fed forward) from the load address queue in the load memory unit; (2) After the LOAD address is sent to the first-level data cache to determine whether it is missing or hit, the missing and hit information is written to the corresponding table entry of the circular queue in the LOAD memory access information tracking and sorting module according to the position of the LOAD address in the load address queue in the load memory unit. (3) The circular queue inside the LOAD memory access information tracking and sorting module has three position indices: the queue head, the queue tail, and the current access entry located between the queue head and the queue tail; the queue head and queue tail are consistent with the queue head and queue tail of the address queue loaded from the load memory unit. Under normal circumstances, without any anomalies or branch prediction errors, when a LOAD instruction is submitted, the queue head is submitted and updated; when a LOAD request is inserted, the queue tail is written and updated. The currently accessed entry, located between the head and tail of the queue, stores the address of the next item to be sent to the prefetcher. The initial position of this entry is the same as that of the head of the queue. (4) In each cycle, the LOAD memory access information tracking and sorting module checks whether there is an address hit flag, an address missing flag, or an address feedforward flag in the current access entry. If at least any of the above three flags are present, the LOAD address and the memory access information of the LOAD instruction PC stored in the current access entry are sent to the LOAD memory access address history buffer and prefetcher, and the position of the current access entry is updated at the same time.

3. The cache access system supporting out-of-order processor data prefetching according to claim 1, characterized in that, The invalidation of the target prefetch address issued by the prefetcher in the target prefetch address buffer is as follows: (1) When the prefetcher generates a new target prefetch address, it determines whether the address is valid from two aspects. First, it checks whether the same address exists in the valid entries in the LOAD memory access address history buffer and the LOAD memory access information tracking and sorting module. If it exists, it means that the target prefetch address has been actually accessed by the processor core recently, and the corresponding data is very likely to be in the cache, so the address is invalid. Second, it checks whether the same address exists in the valid entries in the target prefetch address buffer. If it exists, it means that the target prefetch address has been sent recently and is a duplicate target prefetch address, so the address is invalid. (2) After the target prefetch address generated by the prefetcher is determined to be a valid address, it is inserted into the tail of the queue of the target prefetch address buffer. In each cycle, it checks whether the target prefetch address in each entry of the target prefetch address buffer is the same as the hit address or missing address given by the current level data cache. If the address stored in a certain entry is the same as that, it means that the address is being accessed and the address stored in the entry is no longer timely. The entry is set as an invalid entry and is no longer used for prefetching.

Images