Neural Network Graph Partitioning for Resource-Constrained Hardware
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing graph partitioning methods for neural networks on resource-constrained hardware systems are inefficient, non-scalable, and require real hardware models, leading to sub-optimal performance and high time complexity.
Innovation Solution
A multi-phase neural network graph partitioning system that uses a list-creating unit, node-generating unit, optimizing unit, partition-creating unit, and sub-graph optimizer to segment the graph into sub-graphs based on cost functions, optimizing hardware utilization and meeting hardware limitations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire neural network graph is loaded onto the hardware chip, then complete computation can be performed, but hardware memory limitations are exceeded
Solution Approach 1:
The patent segments the neural network graph into multiple sub-graphs that can be loaded and executed sequentially on the hardware chip. This segmentation allows the system to process large neural networks that exceed the chip's memory capacity by dividing the computation into manageable chunks that fit within available memory resources.
2Manufacturing precision
If graph partitioning is performed manually based on hardware boundaries, then partition accuracy is improved, but system complexity and manual effort increase
Solution Approach 1:
The system automatically performs graph partitioning by analyzing the neural network graph structure and hardware capabilities, eliminating the need for manual configuration. The automated partitioning algorithm generates optimal sub-graph divisions based on computational dependencies and hardware constraints, reducing both manual effort and system complexity while maintaining high partition accuracy.
3Quantity of substance
If the graph is partitioned into many small sub-graphs, then hardware memory constraints are satisfied, but hardware utilization efficiency decreases
Solution Approach 1:
The system dynamically determines the optimal granularity of sub-graph partitioning by analyzing computational patterns and hardware characteristics. Rather than using fixed small partitions, the system adapts partition sizes to maximize hardware utilization while ensuring memory constraints are met, balancing memory usage efficiency with computational throughput.
4Manufacturing precision
If dynamic programming or exhaustive search algorithms are used for graph partitioning, then optimization accuracy is improved, but time complexity increases
Solution Approach 1:
The patent segments the complex optimization problem into smaller sub-problems by dividing the graph into sub-graphs first, then optimizing each sub-graph independently or in groups. This hierarchical segmentation reduces the overall optimization complexity and computation time compared to applying dynamic programming or exhaustive search to the entire graph at once, while still achieving near-optimal results.
Data Source
AI summary
The present invention discloses a graph partitioning system for running neural networks on resource constrained hardware systems. The graph partitioning system used for partitioning a neural network graph into a series of sub-graphs and further allow the multiple sub-graphs to be executed in available hardware subsystems. The system based on cost function as estimated computation time and memory bandwidth of partitioned sub-graphs. The graph partitioning system is a cycle estimation model of hardware that can run fast and parameterize memory latency. The graph partitioning system supports heterogeneous partition for different type accelerators such as CPU, GPU, ASIC. The present invention also discloses a method for partitioning neural network graph in to series of sub-graphs.