Neural Network Accelerator Subgraph Merging for Execution Speed
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network computations are hindered by sequential execution of operators due to data dependencies and significant memory access times, leading to increased execution times in existing hardware accelerators.
Innovation Solution
Implementing a neural network hardware accelerator with merged operator mapping tables that allow for piece-wise polynomial approximation of subgraphs, enabling faster data processing by reducing arithmetic operations and memory access, and dynamically programming these tables to expand operational sequences without rebooting the hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If sequential execution of operators is used in hardware accelerators, then data dependencies are handled correctly, but execution time increases significantly
Solution Approach 1:
The patent merges multiple sequential operators into a single merged operator that can be executed in parallel. The merged operator combines the functionality of multiple operators (e.g., convolution, activation, normalization) into one unified operation, allowing the hardware accelerator to process multiple operations simultaneously without violating data dependencies, thus reducing execution time while maintaining correctness.
Solution Approach 2:
The patent performs preliminary analysis to identify operators that can be merged before execution. The system analyzes the computational graph to determine which operators have compatible data dependencies and can be combined into merged operators. This preliminary action enables the hardware accelerator to prepare parallel execution patterns in advance, reducing runtime execution time while ensuring correct handling of dependencies.
2Adaptability or versatility
If multiple mapping tables are stored in hardware accelerator, then operational sequences are expanded, but hardware resources increase
Solution Approach 1:
The patent implements a universal mapping table structure that can represent multiple operational sequences using a compact encoding scheme. Instead of storing separate mapping tables for each operational sequence, the system uses a single mapping table with additional control bits or parameters that enable the same hardware structure to perform multiple functions. This reduces hardware resource requirements while maintaining the ability to expand operational sequences.
Solution Approach 2:
The patent transitions from storing multiple separate mapping tables in the horizontal dimension to using a single mapping table with vertical dimension expansion through parameterization. By adding control parameters, address bits, or configuration registers to a single mapping table, the system can selectively access different operational sequences without increasing the fundamental hardware resource footprint, thus achieving expansion in a different dimensional space.
3Productivity
If arithmetic operations are reduced through polynomial approximation, then processing speed increases, but computational precision may decrease
Solution Approach 1:
The patent changes the parameter representation of computational operations by replacing exact arithmetic operations with piece-wise polynomial approximations. The system divides the input range into multiple segments and uses different polynomial parameters for each segment, allowing faster computation through simplified mathematical operations while maintaining precision through careful parameter selection and segmentation. This parameter transformation enables speed improvement without significant precision loss.
Solution Approach 2:
The patent applies partial arithmetic operations by using polynomial approximation only for portions of the computational workflow where precision requirements are lower or where the approximation error can be compensated. Critical operations that require high precision are still performed using exact arithmetic, while less critical operations use the faster polynomial approximation. This selective application of approximation maintains overall computational precision while achieving speed improvements in non-critical paths.
Data Source
AI summary
In one example, a method comprises: receiving input codes, wherein the input codes represent a computational dataflow graph; traversing the computational dataflow graph to identify single-entry-single-exit (SESE) subgraphs of the computational dataflow graph, wherein each SESE subgraph has a sequence of nodes comprising a root node and a child node and representing a sequence of element-wise operators, wherein the root node receives a single input tensor, and wherein the child node outputs a single output tensor; determining a merged operator for each SESE subgraph; and generating executable instructions for the computational dataflow graph to be executed by a hardware accelerator having a first execution unit and a second execution unit, wherein the executable instructions comprise first executable instructions for the merged operators targeted at the first execution unit, and second executable instructions for other operators of the computational dataflow graph targeted at the second execution unit.