Tensor Shape Inference in Deep Learning Compilers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training and inferencing using neural networks require significant memory and computing resources, which can be inefficiently utilized due to the need for explicit tensor shape specifications and manual programmer intervention in specifying shapes, leading to increased complexity and resource wastage.
Innovation Solution
A deep learning compiler and tensor shape solver that automatically infers tensor shapes using a shape inference algorithm, reducing the need for explicit shape specifications by dynamically configuring tensor dimensions and generating optimized code for execution on parallel processing units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If explicit tensor shape specifications are required, then programming precision is improved, but device complexity and programmer effort increase
Solution Approach 1:
The system performs self-service by automatically inferring tensor shapes through shape inference algorithms that analyze the computational graph and operation definitions. The compiler automatically determines tensor dimensions without requiring explicit programmer specifications, thereby maintaining programming precision while eliminating the complexity burden on developers.
Solution Approach 2:
A shape inference algorithm acts as an intermediary between the computational graph definition and the execution engine. This intermediary automatically deduces tensor shapes by propagating dimension information through the graph operations, bridging the gap between high-level operation definitions and low-level memory allocation requirements without requiring explicit shape specifications from programmers.
2Measurement precision
If manual shape specification is required, then measurement precision is improved, but loss of time increases
Solution Approach 1:
The shape inference algorithm performs preliminary action by automatically determining all tensor shapes during the compilation phase before execution. By pre-computing dimension information through graph analysis and shape propagation, the system eliminates the need for programmers to spend time on manual shape specifications while ensuring precise shape information is available for optimized code generation.
3Manufacturing precision
If explicit shape specifications are used, then manufacturing precision is improved, but loss of substance increases
Solution Approach 1:
The system employs dynamic shape inference that adapts to the actual computational graph structure and input data characteristics. Rather than requiring fixed explicit shape specifications that may lead to over-allocation, the shape inference algorithm dynamically determines the precise tensor dimensions needed for each operation, optimizing memory resource utilization while maintaining the precision required for efficient neural network execution.
Data Source
AI summary
Apparatuses, systems, and techniques to infer information from one or more sets of data. In at least one embodiment, a processor uses one or more neural networks to infer information from one or more sets of data based, at least in part, on one or more dynamically configurable dimensions of the one or more sets of data.