Parallel Processing Tiles for Machine Learning Hardware
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems using a single processor to perform machine learning operations such as SoftMax, ArgMax, and TopK are inefficient, leading to delays and slower processing speeds due to sequential data processing and inadequate local memory, resulting in increased data transfer latencies.
Innovation Solution
A machine learning hardware architecture utilizing multiple processing tiles that divides data into smaller subsets for parallel processing, allowing each tile to operate locally without relying on external memory, and efficiently streams instructions and data for simultaneous execution of operations like SoftMax, ArgMax, and TopK.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single processor is used to process large amounts of data sequentially, then device complexity is reduced, but processing speed and productivity deteriorate
Solution Approach 1:
The patent divides the processing system into multiple processing elements (PEs) that can operate in parallel. Each PE processes a subset of data independently, transforming a sequential single-processor approach into a parallel multi-processor architecture. This segmentation enables simultaneous execution of operations across multiple data elements, directly addressing the contradiction by improving productivity through parallelization while accepting increased device complexity as a necessary trade-off.
2Loss of time
If a single processor gathers all data for processing, then data management is simplified, but data transfer latency increases
Solution Approach 1:
The patent segments data into smaller portions that are distributed to different processing elements. Instead of gathering all data to a single processor, each PE receives and processes its local data subset, eliminating the need for large-scale data gathering and reducing data transfer latency. This segmentation of data alongside processing elements resolves the contradiction by minimizing time loss while accepting more complex data distribution architecture.
3Productivity
If local memory is insufficient for processing operations, then processing capability is maintained, but additional external memory access is required increasing latency
Solution Approach 1:
The patent assigns smaller data portions to each processing element that fit within individual local memory capacities. By segmenting both the data and the processing workload, each PE can operate entirely within its local memory without requiring external memory access. This resolves the contradiction by maintaining processing capability through parallel operations while eliminating memory access latency through localized data storage.
Data Source
AI summary
A method includes receiving a set of data. The set of data is divided into a plurality of data portions. The method includes transmitting the plurality of data portions to a plurality of processing tiles, wherein each data portion of the plurality of data portions is associated with a processing tile of a plurality of tiles. Each processing tile of the plurality of tiles performs at least one local operation on its respective data portion to form a local result. The method includes exchanging local results between the plurality of processing tiles. Moreover, the method includes calculating a global value based on the local results. The method further includes performing at least one local operation by each processing tile of the plurality of tiles on its respective data portion based on the global value to form a computed result.