Regrained Tile Execution for Runtime Tensor Partitioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Customizing AI applications for different instruction set architectures and processors is time-consuming, costly, and inefficient, particularly when deploying across varied ISAs and processors.
Innovation Solution
A unified programming interface for regrained tile execution that dynamically determines tensor operation sizes and resource allocation at runtime, using a lookup table to optimize code generation based on available hardware resources and conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If code is customized for different ISAs and processors to take advantage of compute features, then execution performance is improved, but development time and cost increase
Solution Approach 1:
The patent creates a unified programming interface that works across multiple ISAs and processor types. The system provides universal code that can execute on different hardware platforms without customization, achieving multi-functionality where the same code base serves diverse computing environments while maintaining good performance through runtime optimizations.
Solution Approach 2:
The system dynamically adapts code execution at runtime based on the target processor's capabilities. Instead of static customization during development, the program dynamically selects and applies optimizations suited to the specific hardware it's running on, eliminating development time costs while preserving performance benefits.
2Power
If code is customized for different ISAs and processors, then execution performance is improved, but deployment complexity increases
Solution Approach 1:
The unified programming interface enables a single codebase to be deployed across multiple ISA types and processor architectures. This universality simplifies deployment by eliminating the need for separate customized versions for different platforms, while the system maintains the ability to optimize performance for each specific hardware type through runtime adaptation.
3Productivity
If tensor operations are optimized for specific tensor sizes, then computational efficiency is improved, but adaptability to different tensor sizes decreases
Solution Approach 1:
The system implements dynamic tile size selection that adapts to the actual tensor dimensions at runtime. Instead of being fixed to specific tensor sizes, the optimization parameters are determined dynamically based on the input tensor characteristics and available hardware resources, maintaining high computational efficiency across varying tensor sizes.
Solution Approach 2:
The system changes optimization parameters such as tile sizes and partitioning strategies based on runtime conditions including tensor dimensions and hardware capabilities. This parameter adaptation allows the same code to achieve high efficiency for different tensor sizes by adjusting the optimization parameters rather than requiring separate customized code for each size.
4Adaptability or versatility
If runtime determination of tensor operation sizes is implemented, then adaptability to different hardware resources is improved, but computational overhead increases
Solution Approach 1:
The system performs preliminary analysis of hardware capabilities and tensor characteristics at the beginning of execution to determine optimal parameters. By doing this setup work early rather than continuously during computation, the system achieves good adaptability to different hardware resources while minimizing ongoing computational overhead during the main computation phases.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Systems, apparatuses and methods may provide for technology that detects a tensor operation in an application, wherein the tensor operation has an unspecified tensor input size, determines the input tensor size at runtime, and selects a partition configuration for the tensor operation based at least in part on the input tensor size and one or more runtime conditions. In one example, the technology searches a lookup table for the input tensor size and at least one of the runtime condition(s) to select the partition configuration.