Neural Network Operator Fusion for On-Chip Storage Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning-based supersampling neural networks face low utilization of multilevel high-speed on-chip storage and inefficient acceleration due to large input, output, and intermediate data sizes, leading to suboptimal performance.
Innovation Solution
The method involves constructing a directed computing graph for a neural network, traversing nodes to identify operators that can be fused based on preset conditions, and generating fusion operators to optimize storage usage and reduce running time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning-based supersampling neural networks are used, then image restoration quality is improved, but the computing amount and space requirements increase significantly
Solution Approach 1:
The patent merges multiple operators (convolution, activation, batch normalization) into fused operators that execute as single computational units. This reduces the overall data size by eliminating redundant intermediate representations and computing steps, directly addressing the issue of large data sizes in deep learning supersampling networks while maintaining image restoration quality.
2Speed
If multilevel high-speed on-chip storage is used for acceleration, then processing speed is improved, but utilization rate remains low due to large data sizes
Solution Approach 1:
By fusing multiple operators into single computational units, the patent reduces the volume of intermediate data that needs to be stored in on-chip memory. This increases storage utilization rate by fitting more effective computational work into the limited high-speed storage capacity, thereby improving both processing speed and productivity simultaneously.
Solution Approach 2:
The patent segments the neural network computation into operator fusion groups that are optimized for on-chip execution. By organizing computations into fused operator units that can be executed within on-chip storage constraints, the system maximizes the use of high-speed storage while maintaining acceleration benefits.
3Adaptability or versatility
If multiple operators are executed separately in the neural network, then operational flexibility is maintained, but overall running time increases
Solution Approach 1:
The patent combines multiple sequential operators into fused operators that execute as single units with shared memory and computation resources. This eliminates the overhead of separate operator execution including memory access between operators and redundant computation setup, significantly reducing running time while the fused operator structure maintains the functional flexibility of the original operator sequence.
Data Source
AI summary
A system for fusing operators of a neural network is included in a combined processing apparatus. The combined processing apparatus includes an interface apparatus and other processing apparatus. A computing processing apparatus interacts with other processing apparatus to jointly complete a computing operation specified by a user. The combined processing apparatus further includes a storage apparatus. The storage apparatus is connected to the apparatus and other processing apparatus, respectively. The storage apparatus is configured to store data of the apparatus and other processing apparatus. A solution of the present disclosure improves efficiency of various operations in data processing fields including, for example, an artificial intelligence field, thus reducing overall overheads and costs of the operations.