Row-Wise Feature Map Partitioning for Balanced Compute Loads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods struggle to efficiently exploit dynamic sparsity in machine learning model processing, leading to unbalanced processing loads, heat distribution issues, and performance degradation due to unpredictable sparsity patterns.
Innovation Solution
The technique involves splitting feature maps row-wise into sub-maps with balanced sparsity, allowing parallel processing by tensor compute units to achieve balanced loading and even heat distribution, thereby improving processing efficiency and avoiding bottlenecks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If feature maps are processed using conventional methods, then processing can be performed, but processing loads become unbalanced and heat distribution becomes uneven
Solution Approach 1:
The feature map is segmented into multiple sub-maps based on sparsity patterns. Each sub-map is then assigned to different compute units for parallel processing. This segmentation resolves the load balancing issue by distributing work based on actual computational requirements rather than uniform distribution, thereby improving both productivity and ease of operation.
2Temperature
If feature maps are processed using conventional methods, then processing can be performed, but heat distribution becomes uneven causing performance degradation
Solution Approach 1:
By segmenting the feature map into sparsity-based sub-maps and distributing them across compute units, the heat generation is naturally balanced. Each compute unit processes sub-maps with comparable sparsity characteristics, leading to uniform heat distribution and preventing thermal hotspots that would cause performance degradation.
Solution Approach 2:
The patent applies local quality by assigning sub-maps with similar sparsity characteristics to specific compute units. This ensures that each compute unit operates under similar thermal conditions, creating uniform heat distribution across the system and maintaining consistent performance across all units.
3Productivity
If feature maps are partitioned row-wise based on sparsity patterns, then parallel processing efficiency is improved, but processing complexity increases
Solution Approach 1:
The sparsity analysis and partitioning of feature maps is performed as a preliminary action before parallel processing begins. By pre-analyzing sparsity patterns and creating the partitioning scheme in advance, the system enables efficient parallel processing without adding complexity during the actual computation phase. The partitioning overhead is paid once upfront, enabling sustained high productivity during processing.
Data Source
AI summary
Certain aspects of the present disclosure provide techniques for partitioning feature maps to improve machine learning model processing. In one aspect, a method, includes partitioning a feature map row-wise into a plurality of feature sub-maps such that: each respective feature sub-map of the plurality of feature sub-maps is defined with respect to a split row determined based on a dense data element count for each row of the feature map; and each feature sub-map of the plurality of feature sub-maps has a same column dimensionality as the feature map; and assigning each of the plurality of feature sub-maps to one of a plurality of tensor compute units and one of a plurality of tensor feature map memory units for processing in parallel.