Dynamic Load Balancing for Heterogeneous Processing Engines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computing systems with heterogeneous processing engines face challenges in maximizing processing throughput due to inefficient distribution of program instructions across different processing engines.
Innovation Solution
A system and method for dynamically balancing instruction execution using a multithreaded SIMD architecture, where a work distribution unit assigns program instructions to heterogeneous processing engines based on a weighted instruction count, ensuring that each engine is fully utilized by executing specific types of instructions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If fixed function computation units or homogeneous processing engines are used, then system simplicity is maintained, but processing throughput cannot be maximized for heterogeneous instruction types
Solution Approach 1:
The patent implements dynamic load balancing by allowing instructions of the second type to be flexibly assigned to either first or second processing engines based on real-time workload conditions. This dynamic assignment mechanism enables the system to adapt to varying instruction mixes and maximize throughput without requiring static specialization, resolving the contradiction between heterogeneity and simplicity.
Solution Approach 2:
The first processing engine is designed with universal capability to execute both first type and second type instructions, while the second processing engine handles second type and third type instructions. This multi-functionality allows the system to process diverse instruction types efficiently without requiring dedicated engines for each instruction type, thereby improving throughput while controlling complexity.
2Productivity
If processing engines are specialized for specific instruction types, then execution efficiency for those instructions improves, but workload distribution becomes unbalanced
Solution Approach 1:
The patent employs a feedback mechanism where the system continuously monitors the workload state of different processing engines and dynamically adjusts the assignment of second type instructions accordingly. This feedback-driven load balancing ensures that specialized engines maintain high execution efficiency while workload is distributed evenly, preventing any single engine from becoming a bottleneck.
Solution Approach 2:
The system changes the assignment parameters for second type instructions based on real-time engine workload conditions. When one engine becomes overloaded, the assignment policy dynamically shifts to prefer the other engine, maintaining balanced workload distribution while preserving the execution efficiency benefits of specialization for each instruction type.
3Productivity
If all processing engines are kept fully utilized, then overall throughput increases, but power consumption and circuitry requirements increase
Solution Approach 1:
The dynamic load balancing mechanism enables the system to achieve high overall throughput by flexibly distributing second type instructions to whichever engine has available capacity. This allows the system to fully utilize existing processing resources without requiring additional engines, thereby improving throughput while avoiding the increased power consumption and circuitry that would result from adding more hardware.
Data Source
AI summary
An embodiment of a computing system is configured to process data using a multithreaded SIMD architecture that includes heterogeneous processing engines to execute a program. The program is constructed of various program instructions. A first type of the program instructions can only be executed by a first type of processing engine and a third type of program instructions can only be executed by a second type of processing engine. A second type of program instructions can be executed by the first and the second type of processing engines. An assignment unit may be configured to dynamically determine which of the two processing engines executes any program instructions of the second type in order to balance the workload between the heterogeneous processing engines.