Tensor Block Scheduling for Balanced Parallel Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In existing data processing systems, the utilization efficiency of processing units is reduced due to varying processing difficulties of different tensor blocks, leading to decreased processing speed as units wait for each other to complete tasks.
Innovation Solution
The system categorizes tensor blocks by attributes and processes blocks with similar processing difficulties concurrently, using separate storage spaces for different types of tensor blocks and assigning them to specific processing units based on determined indexes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If processing units process different tensor blocks concurrently without classification, then parallel processing capability is utilized, but processing units experience idle waiting time due to varying processing difficulties
Solution Approach 1:
The patent segments tensor blocks into different types based on their attributes (e.g., sparsity, data distribution characteristics). Each type is then processed by dedicated processing units, creating homogeneous processing groups that eliminate idle waiting time caused by heterogeneous processing difficulties.
Solution Approach 2:
The patent assigns different processing units to handle different types of tensor blocks based on their specific characteristics. Each processing unit is optimized for its assigned tensor block type, ensuring that processing difficulties are matched across the parallel processing group and eliminating idle waiting time.
2Reliability
If processing units wait for each other to complete tasks, then task completion is ensured, but utilization efficiency of processing units is reduced
Solution Approach 1:
The patent divides tensor blocks into homogeneous groups based on attributes, with each group assigned to specific processing units. This segmentation ensures that all units in a group have similar processing times, allowing reliable task completion without idle waiting, thus maintaining high utilization efficiency.
Solution Approach 2:
The patent changes the parameter of tensor block classification by introducing attribute-based categorization (e.g., sparsity levels, data distribution patterns). This parameter change enables matching processing units to appropriate tensor block types, ensuring both reliable completion and high utilization efficiency.
Data Source
AI summary
A data processing method and apparatus are described. The data processing apparatus obtains an input tensor corresponding to input data. The data processing apparatus determines M1 first-type tensor blocks and M2 second-type tensor blocks. P processing units in the data processing apparatus process the M tensor blocks concurrently. In a first time period, all of the tensor blocks that are processed concurrently by the P processing units are first-type tensor blocks. In a second time period, all of the tensor blocks that are processed concurrently by the P processing units are second-type tensor blocks.