Computing Graph Compilation for Variable Input Ranges
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning frameworks face performance optimization challenges due to variable input sizes in neural networks, leading to inefficient inference phases, as existing solutions are cumbersome and inflexible in setting and managing multiple groups of variable input ranges.
Innovation Solution
A compiling method for computing graphs that supports multiple groups of variable input ranges, allowing flexible configuration and automatic selection of optimal performance optimization graphs at runtime, eliminating the need for manual setup and optimizing redundant steps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If variable input ranges are supported in deep learning frameworks, then adaptability is improved, but performance optimization becomes difficult due to the need to handle multiple input size scenarios
Solution Approach 1:
The patent segments the variable input range support into multiple discrete input size specifications. Each specification corresponds to a pre-compiled computing graph optimized for that specific input size. This segmentation allows the system to maintain high performance for each specific case while collectively supporting variable inputs, resolving the contradiction between adaptability and performance optimization.
2Adaptability or versatility
If multiple groups of variable input ranges are configured, then adaptability is improved, but device complexity increases due to manual setup requirements
Solution Approach 1:
The patent implements self-service by enabling the computing graph compilation system to automatically generate and select optimized computing graphs based on the actual input data size. The system autonomously matches the input size with the appropriate pre-compiled graph without requiring manual configuration or user intervention, thereby supporting multiple variable input ranges while eliminating manual setup complexity.
3Productivity
If performance optimization is pursued for specific input sizes, then inference performance is improved, but ease of operation deteriorates due to manual configuration requirements
Solution Approach 1:
The patent applies preliminary action by pre-compiling multiple computing graphs in advance, each optimized for a specific input size specification. These pre-compiled graphs are stored and ready for use. During runtime, the system simply selects the appropriate pre-compiled graph based on the actual input size, achieving high inference performance without requiring manual configuration at operation time, thus improving ease of operation.
Data Source
AI summary
A compiling method for a computing graph is implemented by a processing apparatus, and a running method for a computing graph is implemented by a computing apparatus. The processing apparatus and the computing apparatus are included in a combined processing apparatus. The combined processing apparatus further includes an interface apparatus. The computing apparatus interacts with the processing apparatus to jointly complete a computing operation specified by a user. The combined processing apparatus further includes a storage apparatus. The storage apparatus is respectively connected to the computing apparatus and the processing apparatus and is configured to store data of the computing apparatus and the processing apparatus. The compiling method and the running method for the computing graph may simplify user operations and improve optimization performance of the computing graph.