Cache Access Circuitry Merging Requests for Many-Core Processors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern many-core computer systems, such as graphics processing units (GPUs), experience inefficient cache utilization due to independent cache access requests from multiple execution units, leading to repeated access of the same cache lines and wasted cache throughput.
Innovation Solution
The implementation merges multiple cache access requests to the same cache line into a single access, optimizing cache throughput by exploiting data locality within and across threads, either locally within an execution cluster or globally across clusters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If each execution unit independently generates cache access requests, then each EU can access cache data autonomously, but cache lines are repeatedly accessed leading to inefficient cache throughput utilization
Solution Approach 1:
The patent merges multiple independent cache access requests from different execution units into a single unified request when they target the same cache line. This is achieved through a cache access circuitry that tracks pending requests and combines them before forwarding to the cache, thereby reducing redundant accesses and improving cache throughput utilization while preserving autonomous access capability for each EU.
2Ease of operation
If vector accesses are handled independently, then each access can be processed separately, but cache throughput is wasted due to partial accesses that could be merged into full accesses
Solution Approach 1:
The patent performs preliminary tracking and merging of vector access requests before they reach the cache. The cache access circuitry maintains a window of pending requests and proactively merges partial vector accesses that target overlapping cache lines, converting them into efficient full cache line accesses before the actual cache operation occurs, thereby eliminating throughput waste.
3Device complexity
If multiple cache access requests to the same cache line are processed separately, then request processing is simple, but redundant accesses reduce cache utilization efficiency
Solution Approach 1:
The patent introduces an intermediary cache access circuitry layer between the execution units and the cache that handles the merging logic. This intermediary tracks pending requests, identifies duplicate cache line accesses, and combines them into single requests, thereby improving cache utilization efficiency while keeping the execution units and cache simple without requiring complex changes to either end.
Data Source
AI summary
Apparatus and method for improved cache utilization and efficiency on a many-core processor. An apparatus comprising: a plurality of execution units to generate cache access requests responsive to executing instructions; a pending request queue to store pending cache access requests generated by the execution units; pending queue management circuitry to compare a current cache access request with entries in the pending request queue to determine whether the current cache access request can be merged with an entry in the pending request queue and, if so, to merge the current cache access request with the entry.