Neural Network Subgraph Recognition for Precompiled Instructions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural network processors face inefficiencies due to programming that does not maximize hardware resource utilization, leading to underutilization and degraded performance.
Innovation Solution
A system that identifies computational subgraphs from a neural network graph and provides pre-compiled executable instructions optimized for the neural network processor, leveraging human experts or machine learning to generate instructions that ignore built-in assumptions, improving efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional programming is used for neural network processors, then the processor can execute neural network computations, but hardware resource utilization is not maximized leading to underutilization and degraded performance
Solution Approach 1:
The patent applies preliminary action by pre-compiling executable instructions for computational subgraphs before runtime. The system identifies computational subgraphs from the computational graph and generates optimized executable instructions in advance, storing them for later retrieval. This eliminates the need for runtime compilation and ensures that hardware resources are optimally scheduled from the start, maximizing resource utilization and avoiding the degradation that occurs with conventional programming approaches.
Solution Approach 2:
The patent segments the computational graph into multiple computational subgraphs, each representing a specific portion of the neural network computation. By dividing the overall computation into discrete subgraphs, the system can generate and store optimized executable instructions for each subgraph independently. This segmentation allows for more granular optimization of hardware resource scheduling and enables efficient retrieval and execution of pre-compiled instructions, thereby improving overall productivity while reducing resource waste.
2Ease of operation
If conventional programming schedules hardware resources, then the processor can operate, but inefficiency is introduced reducing overall performance
Solution Approach 1:
The patent implements self-service by enabling the system to automatically identify computational subgraphs, generate optimized executable instructions, and retrieve pre-compiled code without requiring manual programming intervention for resource scheduling. The system serves itself by maintaining a database of pre-compiled instructions and automatically selecting and executing the appropriate instructions based on the computational graph analysis. This eliminates the inefficiency introduced by conventional programming while maintaining ease of operation, as the optimization process is automated and does not require complex manual scheduling.
3Productivity
If pre-compiled instructions are used for computational subgraphs, then hardware resource usage is maximized, but system complexity increases due to subgraph identification and instruction management
Solution Approach 1:
The patent applies universality by creating a multi-functional system that combines computational graph analysis, subgraph identification, executable instruction generation, and pre-compiled instruction storage within a unified framework. The system serves multiple purposes: it analyzes the computational graph to identify subgraphs, generates optimized instructions for those subgraphs, stores them in a database, and retrieves them during execution. This multi-functional approach consolidates what could be separate complex systems into a single integrated solution, managing complexity while achieving high hardware resource utilization through pre-compiled instructions.
Data Source
AI summary
Systems and methods for providing executable instructions to a neural network processor are provided. In one example, a system comprises a database that stores a plurality of executable instructions and a plurality of subgraph identifiers, each subgraph identifier of the plurality of subgraph identifiers being associated with a subset of instructions of the plurality of executable instructions. The system further includes a compiler configured to: identify a computational subgraph from a computational graph of a neural network model; compute a subgraph identifier for the computational subgraph, based on whether the subgraph identifier is included in the plurality of subgraph identifiers, either: obtain, from the database, first instructions associated with the subgraph identifier; or generate second instructions representing the computational subgraph; and provide the first instructions or the second instructions for execution by a neural network processor to perform computation operations for the neural network model.