Hierarchical Shared Memory for Load-Balancing Compute Units
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing devices with multiple processors or compute units face inefficiencies due to idle units lacking tasks, as current load-balancing mechanisms do not effectively share tasks based on relationships among compute units, leading to suboptimal processing efficiency.
Innovation Solution
The method involves determining sets of compute units and sharing tasks between them by allowing access to private and shared memories, with compute units donating tasks from their private memories to shared memories within their respective sets, enabling task sharing across different levels of compute unit hierarchies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If compute units have private memories with tasks, then task access speed is improved, but task sharing capability deteriorates leading to idle compute units
Solution Approach 1:
The patent merges private memories of multiple compute units into a hierarchical structure with level-one shared memories and level-two shared memories. Compute units can access tasks from their private memories quickly, and when those are empty, they can retrieve tasks from shared memories at different levels, thus combining the speed advantage of private memory with the sharing capability of shared memory.
Solution Approach 2:
The patent implements a nested hierarchical memory structure where level-one shared memories are nested within compute unit sets, and level-two shared memories are nested at a higher level encompassing multiple level-one shared memories. This nested structure allows compute units to access tasks in a hierarchical manner, first from private memory, then level-one shared memory, then level-two shared memory, resolving the contradiction between fast private access and shared availability.
2Productivity
If tasks are shared across all compute units, then compute unit utilization is improved, but synchronization complexity increases
Solution Approach 1:
The patent segments the compute units into multiple sets, with each set having its own level-one shared memory. This segmentation allows task sharing to be localized within each set, reducing the synchronization complexity compared to global task sharing across all compute units. The hierarchy further segments management by allowing independent operation at different levels.
Solution Approach 2:
The patent introduces a hierarchical dimension to task sharing by creating multiple levels of shared memories (level-one and level-two). This dimensional approach to organization allows the system to manage task sharing complexity by distributing the sharing burden across different hierarchical levels, where each level handles a subset of compute units, thereby reducing overall synchronization complexity while maintaining high utilization.
3Adaptability or versatility
If compute units access tasks from shared memories, then task availability is improved, but memory access time increases
Solution Approach 1:
The patent applies local quality by making each compute unit's private memory the primary (fastest) access location, with level-one shared memories providing faster access for nearby compute units in the same set, and level-two shared memories providing access for all compute units. This localized quality gradient ensures that most task accesses remain fast while improving overall availability through the hierarchy.
Solution Approach 2:
The patent implements preliminary action by allowing compute units to donate tasks to their level-one shared memory and level-two shared memory in advance. This preliminary donation ensures that tasks are pre-positioned in the hierarchical memory structure, so when a compute unit needs a task, it is already available in the shared memories, reducing access time while maintaining high availability.
Data Source
AI summary
Sharing tasks among compute units in a processor can increase the efficiency of the processor. When a compute unit does not have a task in its task memory to perform, donating tasks from other compute units can prevent the compute unit from being idle while there is task in other parts of the processor. It is desirable to share tasks among compute units that are within defined scopes of the processor. Compute units may share tasks by allowing other compute units to access their private memory, or by donating tasks to a shared memory.