Tensor Shape Inference for Neural Network Resource Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training and inferencing processes for neural networks require significant memory and computing resources, which are not efficiently utilized due to the need for explicit tensor shape specifications and manual shape constraints, leading to increased complexity and programmer effort.
Innovation Solution
A deep learning compiler that uses a tensor shape solver to infer tensor shapes automatically by replacing symbolic dimensions with specific values, reducing programmer effort and enabling efficient resource utilization through implicit shape representation and constraint-based inference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If explicit tensor shape specifications are required, then memory and computing resources can be allocated, but programmer effort and complexity increase significantly
Solution Approach 1:
The system performs self-service by automatically inferring tensor shapes through constraint propagation and solving mechanisms. The shape inference engine autonomously determines tensor dimensions by analyzing operational constraints and data flow relationships without requiring explicit programmer specifications, thereby reducing programming complexity while maintaining resource allocation efficiency
Solution Approach 2:
The patent applies preliminary action by performing shape inference during the compilation phase before actual execution. The compiler analyzes the computational graph, propagates shape constraints, and resolves tensor dimensions in advance, allowing the system to prepare resource allocation plans beforehand without burdening the programmer with explicit shape specifications
2Ease of operation
If manual shape constraints are used, then tensor operations can be performed, but the process requires significant programmer effort and time
Solution Approach 1:
The shape inference system operates autonomously to determine tensor shapes by analyzing operational constraints and propagating shape information through the computational graph. This self-service mechanism eliminates the need for programmers to manually specify shapes, significantly reducing programmer time while maintaining ease of tensor operation execution
Solution Approach 2:
The system implements feedback mechanisms where shape information is propagated bidirectionally through the computational graph. Constraints from operations feed back to refine shape inferences, and inferred shapes feed forward to enable subsequent operations. This automated feedback loop replaces manual constraint specification, reducing programmer effort and time
3Adaptability or versatility
If symbolic dimensions are used, then flexibility is maintained, but resource utilization efficiency decreases
Solution Approach 1:
The patent transforms symbolic dimension parameters into concrete values through constraint solving. The system maintains symbolic representations during analysis to preserve flexibility, then applies parameter changes by resolving these symbols to specific dimension values once constraints are satisfied, thereby achieving both adaptability and resource utilization efficiency
Solution Approach 2:
The compiler performs preliminary shape resolution by analyzing constraints and determining concrete tensor shapes before code generation. This preliminary action allows the system to maintain symbolic flexibility during compilation while producing optimized code with concrete dimension values, improving resource utilization efficiency without sacrificing adaptability
4Device complexity
If automatic shape inference is implemented, then programmer effort is reduced, but computing overhead during compilation increases
Solution Approach 1:
The shape inference process is segmented into distinct phases: constraint extraction, constraint propagation, and constraint solving. Each phase processes a specific subset of the computational graph and constraints, dividing the overall computing workload into manageable segments. This segmentation reduces peak computing resource requirements during compilation while maintaining automatic shape inference capabilities
Data Source
AI summary
Apparatuses, systems, and techniques to determine dimensions of one or more sets of data. In at least one embodiment, a processor causes one or more dimensions of one or more sets of data to be determined using one or more dimensional constraints of the one or more sets of data.