Subcache Affinity for Multi-Processor Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In data centers, existing technologies face challenges in efficiently managing subcache affinity and latency in multi-processor environments, where multiple applications with different Quality of Service (QoS) requirements run concurrently, leading to suboptimal data retrieval and storage due to the proximity of subcaches to compute units.
Innovation Solution
A method and apparatus that prioritize space allocation in subcaches based on proximity to compute units, allowing victim data to be injected into the nearest available subcache and optimizing data retrieval by sending Tag Lookup Request messages directly to the nearest subcache, thereby reducing latency and improving data allocation efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If victim data is allocated to subcaches in arbitrary order, then space allocation is simple, but latency increases due to distant subcache access
Solution Approach 1:
The patent applies local quality by making subcaches have different priorities based on their physical proximity to compute units. The nearest subcache receives highest priority for victim data allocation, while farther subcaches receive lower priority. This creates a gradient of quality in space allocation that directly reduces access latency by preferentially using closer storage locations.
Solution Approach 2:
The patent implements preliminary action by pre-establishing affinity relationships between compute units and subcaches based on physical proximity before data allocation occurs. The system pre-determines which subcaches are nearest to which compute units and uses this pre-computed affinity information to guide victim data allocation decisions, avoiding the need for complex real-time distance calculations.
2Loss of time
If Tag Lookup Request messages are sent to all subcaches, then data retrieval reliability is high, but latency increases due to broadcasting to distant subcaches
Solution Approach 1:
The patent applies local quality by directing Tag Lookup Request messages preferentially to nearer subcaches based on their affinity with the requesting compute unit. The system sends requests first to the nearest subcache, then to progressively farther subcaches if needed, creating a localized search pattern that reduces average lookup latency while maintaining reliability through the hierarchical search approach.
Solution Approach 2:
The patent implements preliminary action by pre-establishing affinity relationships between compute units and subcaches based on physical proximity before data allocation occurs. The system pre-determines which subcaches are nearest to which compute units and uses this pre-computed affinity information to guide victim data allocation decisions, avoiding the need for complex real-time distance calculations.
3Speed
If subcaches are physically located near compute units, then access speed improves, but data allocation efficiency decreases due to space fragmentation
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the priority parameter of subcaches based on their physical distance from compute units. The system changes the allocation priority parameter from a uniform value to a distance-dependent value, where nearer subcaches have higher priority. This parameter transformation enables the system to exploit physical proximity for faster access while maintaining high allocation efficiency through priority-based victim data placement.
Data Source
AI summary
A method and apparatus for controlling affinity of subcaches is disclosed. When a core compute unit evicts a line of victim data, a prioritized search for space allocation on available subcaches is executed, in order of proximity between the subcache and the compute unit. The victim data may be injected into an adjacent subcache if space is available. Otherwise, a line may be evicted from the adjacent subcache to make room for the victim data or the victim data may be sent to the next closest subcache. To retrieve data, a core compute unit sends a Tag Lookup Request message directly to the nearest subcache as well as to a cache controller, which controls routing of messages to all of the subcaches. A Tag Lookup Response message is sent back to the cache controller to indicate if the requested data is located in the nearest sub-cache.