Neural Network Layer Fusion to Reduce Data Transfer Costs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network execution is inefficient due to complex code, which can be improved through compiler optimization.
Innovation Solution
Layer fusion is applied to combine operations in neural networks, reducing data transfer and manipulation costs by identifying fusible edges and estimating edge costs, thereby optimizing execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If layer fusion is applied to combine operations, then execution efficiency is improved, but code complexity increases
Solution Approach 1:
The patent applies layer fusion to combine multiple neural network layers into a single fused layer, merging operations such as convolution, batch normalization, and activation functions that were previously executed as separate operations. This consolidation reduces the number of sequential operations and data transfers required, thereby improving execution efficiency and reducing overall code complexity despite the increased operational density within the fused layer.
2Loss of energy
If operations are combined through layer fusion, then data transfer costs are reduced, but computational density increases
Solution Approach 1:
By merging multiple operations into a single fused layer, the patent eliminates intermediate data transfers between separate layers, significantly reducing data transfer costs and memory bandwidth requirements. The fused layer performs all necessary computations (convolution, normalization, activation) in-place, avoiding repeated reads and writes to memory, thereby reducing energy consumption associated with data transfer.
Solution Approach 2:
The patent changes the operational parameters by executing multiple computations with different data types and precision levels within the same fused layer. This allows optimization of computational density by selecting appropriate precision for each operation while maintaining overall efficiency, balancing computational requirements with memory access patterns.
Data Source
AI summary
Apparatuses, systems, and techniques to perform a neural network. In at least one embodiment, a neural network is performed by at least combining nodes of a graph based, at least in part, on computing resources to perform operations corresponding to the nodes of the graph.