GUI Tensor Graph Tiling for Cache-Fit AI Operation Subtasks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The complexity of deep neural networks makes manual specification of tiling in text-based environments difficult, leading to inefficient cache usage and increased DRAM utilization due to large tensor sizes that exceed cache limits, affecting processing efficiency.
Innovation Solution
A tensor graph configuration system and method using a graphical user interface (GUI) to intuitively adjust tensor graphs, allowing users to visually select tiling and pipelining mechanisms, including wake-up and wait signals, to split operations into sub-tasks and optimize cache usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If a large tensor size is used to process complex neural network operations, then the computational capability is improved, but the cache utilization deteriorates leading to increased DRAM utilization
Solution Approach 1:
The patent applies segmentation by dividing a large tensor into multiple smaller sub-tensors through tiling. The tensor graph configuration system automatically splits operation tasks into sub-tasks that process smaller tensor segments, allowing these sub-tensors to fit within the limited cache memory while maintaining the overall computational capability for complex neural network operations.
2Manufacturing precision
If manual tiling specification is used in text-based environments, then the configuration precision can be controlled, but the ease of operation deteriorates due to complexity
Solution Approach 1:
The patent replaces the manual text-based configuration mechanism with an automated graphical user interface system. The GUI allows users to visually configure tensor graphs through intuitive graphical elements rather than text commands, while the system automatically generates the precise tiling configurations, eliminating the complexity of manual specification while maintaining configuration precision.
Solution Approach 2:
The tensor graph configuration system performs self-service by automatically determining the tiling configuration based on the tensor graph and operational requirements. The system autonomously analyzes the tensor graph, identifies optimal tiling strategies, and generates the configuration without requiring manual intervention, thereby improving ease of operation while maintaining precision.
3Quantity of substance
If tiling configuration is optimized for cache fitting, then the cache utilization is improved, but the device complexity increases
Solution Approach 1:
The system performs self-service by automatically determining optimal tiling configurations that fit cache constraints. The tensor graph configuration system autonomously analyzes tensor sizes, cache capacities, and computational requirements to generate appropriate tiling strategies, eliminating the need for manual configuration while achieving optimal cache utilization.
Solution Approach 2:
The system employs feedback mechanisms where the tensor graph configuration system continuously monitors cache utilization and computational performance, then automatically adjusts tiling configurations accordingly. This closed-loop approach allows the system to optimize cache fitting while managing complexity through adaptive rather than static configuration.
Data Source
AI summary
A tensor graph configuration method includes providing a tensor graph comprising a plurality of operation tasks through a graphical user interface (GUI), determining a sub-graph from the tensor graph based on commands input through the GUI, wherein at least one operation task is within the sub-graph, determining a tiling configuration for the sub-graph according to multiple target tensor sizes, wherein the multiple target tensor sizes are set for a final operation task in the sub-graph by a user through the GUI, and generating a tiled tensor graph according to the tiling configuration, wherein the tiled tensor graph at least comprises a plurality of operation sub-tasks split from a respective operation task in the sub-graph through the tiling.