GPU Program Control via Memory-Aware Execution Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for executing multiple programs on a GPU often result in memory shortages, leading to errors and inefficiencies, as they do not effectively manage memory usage when programs are run simultaneously.
Innovation Solution
A program control device and method that convert programs into intermediate representations with memory information, determining an execution order to keep memory usage below a threshold, ensuring simultaneous execution without memory overflow.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple programs are executed simultaneously on a GPU, then program execution efficiency and productivity are improved, but memory usage exceeds the GPU memory threshold causing execution errors
Solution Approach 1:
The system performs preliminary actions by converting programs into intermediate representations before execution, analyzing their memory requirements in advance, and determining an execution order that prevents memory overflow. This allows multiple programs to be scheduled for simultaneous execution while ensuring GPU memory thresholds are not exceeded.
Solution Approach 2:
The system dynamically determines the execution order of intermediate representations based on real-time memory usage monitoring. When GPU memory approaches the threshold, the system adjusts the execution sequence by selecting different intermediate representations, enabling flexible memory management during concurrent program execution.
2Power
If GPU memory is allocated for multiple programs, then computing capability is enhanced, but the risk of memory overflow increases causing program termination
Solution Approach 1:
The system implements feedback mechanisms by monitoring GPU memory usage during program execution and adjusting the execution order of intermediate representations accordingly. When memory usage approaches the threshold, the system receives feedback and reorders execution to maintain stable operation without program termination.
Solution Approach 2:
Before execution begins, the system converts programs into intermediate representations and analyzes their memory footprints in advance. This preliminary analysis allows the system to predict potential memory conflicts and determine a safe execution order that ensures reliable operation throughout execution.
3Adaptability or versatility
If time-shared GPU execution is used, then hardware resource utilization is improved, but execution speed decreases due to sequential processing
Solution Approach 1:
The system performs preliminary conversion of programs into intermediate representations and determines an optimized execution order before actual execution begins. This allows the GPU to execute multiple programs in parallel simultaneously rather than sequentially, maintaining high execution speed while achieving effective hardware resource utilization through proper scheduling.
Data Source
AI summary
A program control device includes a conversion unit which converts multiple programs to be simultaneously executed by an accelerator into intermediate representations indicating computation operations to be executed by programs and memory information indicating an amount of memory required by data used in the computation operations, respectively, and a determination unit which determines an execution order of multiple intermediate representations so that an amount of memory usage used by the accelerator when the accelerator executes the multiple programs simultaneously is below a threshold value of the memory based on the converted multiple intermediate representations and multiple memory information.