Neural Network Graph Management via Subgraph Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network systems face inefficiencies in processing large calculations due to the lack of effective management of calculation processing graphs across heterogeneous resources, leading to suboptimal performance and increased processing times.
Innovation Solution
A neural network system that employs a task manager to allocate subgraphs to appropriate resources and independent compilers for parallel compilation, allowing each subgraph to be processed through separate calculation paths, thereby optimizing resource utilization and reducing compilation time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single compiler is used to process the entire calculation processing graph, then the system structure is simple, but the compilation time increases and processing efficiency decreases
Solution Approach 1:
The calculation processing graph is divided into multiple subgraphs, each of which can be compiled independently by separate compilers. This segmentation allows parallel compilation processes to occur simultaneously, significantly reducing total compilation time while maintaining manageable system complexity through modular architecture.
2Loss of time
If multiple compilers are used for parallel compilation, then compilation time is reduced, but resource management complexity increases
Solution Approach 1:
A resource manager acts as an intermediary component that coordinates between multiple compilers and heterogeneous resources. It allocates subgraphs to appropriate compilers, manages resource distribution, and handles compilation scheduling, thereby reducing the complexity burden from individual compilers while enabling efficient parallel processing.
Solution Approach 2:
The system dynamically assigns subgraphs to compilers based on resource availability and compilation priorities. The resource manager can adjust compilation schedules and resource allocation in real-time, allowing the system to adapt to changing conditions and optimize compilation throughput without fixed rigid structures.
3Adaptability or versatility
If heterogeneous resources are used for processing, then processing capability is enhanced, but resource allocation complexity increases
Solution Approach 1:
The resource manager implements a universal interface layer that handles multiple types of heterogeneous resources (GPUs, CPUs, FPGAs, etc.) through a common allocation mechanism. This allows the system to support diverse processing capabilities while maintaining a unified resource management approach, reducing allocation complexity despite the variety of available resources.
Data Source
AI summary
Provided are a system for managing a calculation processing graph of an artificial neural network and a method of managing a calculation processing graph by using the system. A system for managing a calculation processing graph of an artificial neural network run by a plurality of heterogeneous resources includes: a task manager configured to allocate the plurality of heterogeneous resources to a first subgraph and a second subgraph that are to be run, the first subgraph and the second subgraph being included in the calculation processing graph; a first compiler configured to compile the first subgraph to be executable on a first resource among the plurality of heterogeneous resources; and a second compiler configured to compile the second subgraph to be executable on a second resource among the plurality of heterogeneous resources, wherein the first subgraph and the second subgraph are respectively managed through separate calculation paths.