Neural Network Graph Fusion for Lower Memory and Compute Load
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training and inferencing using neural networks require significant memory, time, and computing resources, which can be optimized.
Innovation Solution
A deep learning compiler combines independent operations in neural networks using fusion rules to generate optimized code for execution on parallel processing units, reducing redundant computations and improving resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If independent operations are executed separately in neural networks, then operational flexibility is maintained, but memory and computing resource consumption increases
Solution Approach 1:
The patent combines multiple independent operations into fused operations that execute as unified computational units. This merging reduces the number of separate memory accesses and computational overhead, thereby decreasing resource consumption while maintaining the functional capabilities of the original independent operations through careful preservation of operation semantics and data flow.
2Manufacturing precision
If multiple independent operations are performed sequentially, then operational precision is maintained, but execution time increases
Solution Approach 1:
The fusion of multiple operations into single computational units eliminates intermediate memory I/O operations and reduces computational overhead, significantly decreasing execution time. The patent maintains operational precision by preserving the mathematical equivalence of the fused operations and ensuring correct data dependencies are maintained throughout the fusion process.
3Productivity
If operations are fused together, then resource utilization improves, but operational complexity increases
Solution Approach 1:
The patent applies fusion rules that systematically combine compatible operations while maintaining a clear representation of the computational graph. This approach improves resource utilization by reducing redundant computations and memory accesses, while the rule-based fusion methodology keeps the transformation process manageable and the resulting operational complexity controlled through structured graph manipulation.
Data Source
AI summary
Apparatuses, systems, and techniques to combine operations. In at least one embodiment, a processor causes two or more operations in a graph to be combined based, at least in part, on another combination of two or more independent operations.