Data-Driven Automatic Operator Fusion in Computational Graphs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for operator fusion in neural networks rely on manual analysis, which is inefficient and cumbersome, especially for large-scale networks with numerous operators, leading to excessive resource consumption and delayed computing times.
Innovation Solution
An automatic fusion method for operators in computational graphs that uses data-driven performance evaluation models to search for optimal fusion ways, defining control data and employing a performance evaluation model to determine the best fusion strategy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If manual analysis method is used for operator fusion, then fusion performance can be optimized, but the process becomes extremely cumbersome and inefficient for large-scale networks
Solution Approach 1:
The patent replaces manual mechanical analysis with an automated system that uses computational graphs and programmatic operator fusion. The system automatically identifies fusion opportunities, evaluates performance impacts, and executes fusion operations through software algorithms rather than manual human analysis, thereby resolving the contradiction between optimization precision and processing efficiency.
Solution Approach 2:
The system enables self-service automation where the computational graph system itself performs the fusion analysis and optimization without requiring manual intervention. The automated operator fusion mechanism allows the system to analyze its own structure, identify fusion candidates, and execute optimizations autonomously, transforming a manual process into a self-service automated one.
2Manufacturing precision
If manual operator fusion analysis is performed, then fusion optimization is achieved, but resource consumption increases and computing time is delayed
Solution Approach 1:
The patent implements preliminary action by pre-defining fusion strategies, fusion criteria, and performance evaluation models before actual fusion operations are needed. The system prepares fusion templates, identifies potential fusion candidates in advance, and establishes optimization rules that can be executed automatically, thereby reducing the time required for actual fusion optimization operations.
Solution Approach 2:
The system substitutes manual time-consuming analysis with automated computational algorithms that quickly evaluate fusion candidates and execute optimizations. By replacing manual inspection and analysis with programmatic approaches, the system achieves fusion optimization significantly faster, resolving the contradiction between optimization quality and time consumption.
3Manufacturing precision
If manual analysis is used for operator fusion, then fusion benefits can be maximized, but the complexity and burden on operators increases
Solution Approach 1:
The system performs self-service automation where the computational graph framework automatically handles fusion analysis, candidate identification, and optimization execution without requiring operator intervention. This transforms a complex manual process into an automated self-service mechanism, reducing the complexity burden on operators while maintaining high fusion benefits through systematic algorithmic optimization.
Solution Approach 2:
The patent introduces an intermediary automated fusion management layer that mediates between the computational graph structure and the fusion optimization process. This intermediary system handles the complexity of fusion analysis, evaluation, and execution automatically, shielding operators from direct complexity while enabling maximized fusion benefits through sophisticated automated algorithms.
Data Source
Figure 1~2
Figure 3a~3b
Figure 4~5
AI summary
This disclosure discloses a method for automatic fusion of operators in a computational graph, a computing device, and related products. The computing device may be included in a combination processing device, which may also include an interface device and other processing devices. The computing device interacts with other processing devices to jointly complete computing operations specified by the user. The combined processing devices may also include a storage device, which is connected to the computing device and other processing devices respectively, for storing data of the computing device and other processing devices. The solution disclosed herein provides a solution for automatic fusion of operators in a computational graph, which may automatically search for optimized operator fusion solutions without the need for manual analysis, thereby improving the efficiency of fusion analysis.