MLA Tensor Partitioning for Edge Data Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning networks require significant data transfer and computations, leading to high memory usage and communication bandwidth, which is challenging to manage efficiently, especially when implemented on edge devices with limited resources.
Innovation Solution
A machine learning accelerator (MLA) uses statically scheduled data transfer instructions that coordinate concurrent data transfers between processing elements (PEs), partitioning tensors into slices and allocating them to PEs to reduce memory usage and increase parallelism, allowing for efficient data routing and computation execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If machine learning networks are implemented with significant data transfer and computations, then computational capability is improved, but memory usage and communication bandwidth increase
Solution Approach 1:
The patent divides tensors into multiple slices and distributes them across different processing elements (PEs). Each PE handles a specific slice, enabling parallel processing while reducing the memory burden on individual elements. This segmentation allows the system to process large datasets without requiring each PE to store the entire dataset in memory.
Solution Approach 2:
The patent introduces a spatial dimension by organizing PEs in a mesh topology and using coordinate-based addressing (row and column indices). Data is routed through intermediate PEs based on source and destination coordinates, transforming the memory access problem into a spatial routing problem that can be solved through coordinated data transfer instructions.
2Productivity
If machine learning networks are implemented with significant data transfer and computations, then computational capability is improved, but communication bandwidth requirements increase
Solution Approach 1:
By segmenting data into slices and distributing them across multiple PEs, the patent reduces the communication bandwidth required at any single point in the network. Each PE communicates only with neighboring PEs for its specific slice, rather than requiring all PEs to access all data simultaneously.
Solution Approach 2:
The patent implements continuous data flow through the PE mesh using coordinated transfer instructions. Data moves continuously from source PEs through intermediate PEs to destination PEs without interruption, maximizing the utilization of available communication bandwidth and avoiding idle periods.
3Productivity
If data is transferred between processing elements, then computation parallelism is increased, but coordination complexity increases
Solution Approach 1:
Each PE autonomously executes instructions and manages its own data slice without requiring centralized control for every operation. The coordinated transfer instructions enable PEs to self-organize data movement based on their positions in the mesh and their computational requirements, reducing the need for complex external coordination mechanisms.
Data Source
AI summary
A machine learning accelerator (MLA) implements a machine learning network (MLN) by using data transfer instructions that coordinate concurrent data transfers between processing elements. A compiler receives a description of a machine learning network and generates the computer program that implements the MLN. The computer program contains instructions that will be run on PEs of the MLA. The PEs are connected by data transfer paths that are known to the compiler. The computations performed by the PEs may require data stored at other PEs. The compiler coordinates the data transfers to avoid conflicts and increase parallelism.