Flattened Data Structures for Faster Pointer-Chase Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing compiler optimizations for improving data locality in computer programs, such as structure peeling and prefetching, are limited in effectiveness and often require hardware-specific tuning, failing to address the inefficiencies in memory access due to pointer chasing.
Innovation Solution
A method and apparatus that flatten multi-level data structures by identifying chains of pointers and replacing them with flatter data structures, reducing the number of memory accesses and improving data locality through a cost analysis and cache allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multi-level data structures with pointer chains are used, then program flexibility and data organization are improved, but memory access time increases and data locality deteriorates
Solution Approach 1:
The patent segments the pointer chain access path by introducing a cache structure that stores intermediate pointer values. This divides the multi-level indirect access into two parts: (1) direct access to cache for intermediate pointers, and (2) access to target data. The cache structure breaks the long pointer chain into shorter segments, improving data locality for frequently accessed pointers.
Solution Approach 2:
The patent performs preliminary action by pre-loading pointer values into the cache structure before they are needed for data access. The cache is populated with pointer values from multi-level data structures in advance, so that when data access is required, the intermediate pointers are already available in fast memory, eliminating the need for immediate multi-level indirect access.
2Adaptability or versatility
If pointer chains are used to reference stored information, then data structure flexibility is improved, but the number of cache misses increases
Solution Approach 1:
The patent introduces a cache structure as an intermediary between the multi-level data structures and the CPU. This cache acts as a mediator that stores frequently accessed pointer values, allowing the system to maintain flexible multi-level data structures while improving cache hit rates. The intermediary cache bridges the gap between the flexible but cache-unfriendly pointer chains and the cache-oriented memory access pattern required for high performance.
3Adaptability or versatility
If complex pointer chains are used, then program functionality is improved, but device complexity increases
Solution Approach 1:
The patent creates a simplified copy of the pointer chain structure in the cache. Instead of maintaining the full multi-level pointer structure in fast memory, the system copies only the essential pointer values into the cache structure. This copying approach preserves the functional capabilities of the complex data structures while reducing the complexity of the memory access pattern, as the cache contains a simplified representation that enables direct access.
Data Source
AI summary
A method and apparatus for computer operation improvement by flattening multi-level data structures to optimize pointer chase is provided. Within computer program code, one or more chains of multiple pointers which ultimately reference stored information to be processed are identified. These chains are analyzed to determine the feasibility of replacing the chains with a flatter data structure. In the flatter data structure, the stored information is accessible more directly. For example the data may be stored in a single array by which it can be accessed directly. When feasible, a chain is replaced with such a data structure and the computer program code is adjusted to implement the improved data structure in place of the associated chain. Candidate chains can be identified or prioritized for example on the basis of the expected impact to computer performance, e.g. in terms of operating speed or resource usage.