Intermediate Representation Generation for Multi-Accelerator Programs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing scale of deep learning models requires multiple accelerators for operation handling, necessitating manual configuration of connection relationships and communication frameworks, which is cumbersome and inefficient.
Innovation Solution
A method and system for generating an intermediate representation of a program that automatically converts it for execution on multiple accelerators, optimizing data distribution and parallel operation without manual intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If multiple accelerators are used to handle increasing operations, then processing capability is improved, but configuration complexity increases
Solution Approach 1:
The system automatically generates intermediate representations and performs program conversion without requiring manual user configuration. The accelerator selection and program adaptation are performed autonomously by the system based on the input program and available accelerators, eliminating the need for users to manually configure connection relationships and communication frameworks.
Solution Approach 2:
An intermediate representation layer is introduced between the original program and the accelerator execution. This intermediate representation serves as a mediator that automatically adapts the program to suit multiple accelerators, handling the complexity of acceleration selection and program conversion without affecting the user's original program code.
2Adaptability or versatility
If manual settings are performed for multiple accelerator connections, then program adaptability is improved, but time consumption increases
Solution Approach 1:
The system pre-establishes a library of available accelerators and their capabilities before program execution. When a program needs to be executed, the system automatically selects appropriate accelerators from this pre-configured library and generates the necessary intermediate representations, eliminating the need for manual configuration time while maintaining high adaptability.
Solution Approach 2:
The system performs automatic accelerator selection and program conversion without requiring user intervention. Users simply provide the original program and the system autonomously handles all adaptation tasks, including identifying suitable accelerators, generating intermediate representations, and preparing the program for execution.
3Productivity
If original program is modified for multiple accelerators, then execution efficiency is improved, but development complexity increases
Solution Approach 1:
The intermediate representation acts as a mediator layer that separates the original program from the accelerator-specific implementation details. Users write their programs in the original framework without modification, and the intermediate representation layer automatically translates and adapts them for multiple accelerators, preserving both the original program's simplicity and the execution efficiency on different hardware.
Solution Approach 2:
The system divides the program execution into distinct segments: the original program remains unchanged, the intermediate representation generation is performed automatically, and the acceleration-specific optimizations are applied at the intermediate level. This segmentation allows users to focus only on the high-level program logic while the system handles the complex acceleration adaptation separately.
Data Source
AI summary
A method for generating an intermediate representation for a program for execution on an accelerator is executed by one or more processors, and includes hooking information on instruction from a program, determining whether the hooked information on instruction is associated with an accelerator, if it is determined that the information on instruction is associated with the accelerator, generating a first intermediate representation for the instruction using information on input and output data and information on instruction included in the instruction, and generating a second intermediate representation for the program for one or more accelerators using the first intermediate representation, and the first intermediate representation and the second intermediate representation include a plurality of data nodes, one or more operation nodes, and a plurality of edges indicating an input and output relationship between the plurality of data nodes and the one or more operation nodes.