Method and apparatus for prefetching recursive data structures

Abstract

Computer systems are typically designed with multiple levels of memory hierarchy. Prefetching has been employed to overcome the latency of fetching data or instructions from or to memory. Prefetching works well for data structures with regular memory access patterns, but less so for data structures such as trees, hash tables, and other structures in which the datum that will be used is not known a priori. The present invention significantlyA system and method is provided that increases the cache hit rates of many important data structure traversals, and thereby the potential throughput of the computer system and application in which it is employed. The inventionThis is applicable to those data structure accesses in which the traversal path is dynamically determined. The invention does thisThis is done by aggregating traversal requests and then pipelining the traversal of aggregated requests on the data structure. Once enough traversal requests have been accumulated so that most of the memory latency can be hidden by prefetching the accumulated requests, the data structure is traversed by performing software pipelining on some or all of the accumulated requests. As requests are completed and retired from the set of requests that are being traversed, additional accumulated requests are added to that set. This process is repeated until either an upper threshold of processed requests or a lower threshold of residual accumulated requests has been reached. At that point, the traversal results may be processed.

Description

[0001]This application claims benefit of Ser. No. 60 / 174,745 a provisional application filed Jan. 3, 2000 and claims benefit of provisional application Ser. No. 60 / 174,292 filed Jan. 3, 2000.FIELD OF THE INVENTION[0002]This invention addresses the problem of prefetching indirect memory references commonly found in applications employing pointer-based data structures such as trees and hash tables. More specifically, the invention relates to a method for pipelining transactions on these data structures in a way that makes it possible to employ data prefetching into high speed caches closer to the CPU from slow memory. It further specifies a means of scheduling prefetch operations on data so as to improve the throughput of the computer system by overlapping the prefetching of future memory references with the execution of previously cached data.[0003]1. Background of the Invention[0004]Modern microprocessors employ multiple levels of memory of varying speeds to reduce the latency of re...

AI Technical Summary

Benefits of technology

The present invention improves the performance of computer systems and applications by increasing the cache hit rates of data structure traversals. This is achieved by aggregating and pipelining traversal requests, which helps to hide most of the memory latency. The invention can be used for data structure accesses where the traversal path is dynamically determined. The process is repeated until a certain threshold of processed requests is reached, at which point the results can be processed. Overall, the invention enhances the potential throughput of computer systems and applications.

Problems solved by technology

The more time that expires before a stalled thread is scheduled to execute again, the greater the likelihood that one of the other threads has caused a future operand of the stalled thread to be evacuated from the cache, thereby increasing the miss rater, and so creating a vicious cycle.

Early binding has the drawback that it is difficult to maintain program correctness in pointer based codes because loads cannot be moved ahead of a store unless it is certain that they are to different memory locations.

Memory disambiguation is a difficult problem for compilers to solve, especially in pointer-based codes.

Images