ML Network Layout Splits for Low-Latency Edge Data Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning networks require significant computational resources and data transfers, which are challenging to implement efficiently on edge devices due to latency and power consumption issues, especially when input sources are remotely located from compute facilities.
Innovation Solution
A method for optimizing data transfers in machine learning networks by partitioning tensors into slices for computation by processing elements (PEs) and scheduling data transfers using an autosplit engine, which determines optimal splits and allocates slices to PEs to minimize latency and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If machine learning networks are implemented in cloud compute facilities with significant resources, then computational performance is improved, but latency and power consumption increase due to remote data sources
Solution Approach 1:
The patent segments the machine learning network into multiple processing elements distributed across edge devices. Each processing element handles a portion of the computation locally, eliminating the need to transfer all data to a centralized cloud facility. This segmentation enables parallel processing at the edge, reducing latency while maintaining computational performance.
Solution Approach 2:
The patent introduces a spatial dimension by distributing processing across multiple edge devices rather than concentrating computation in a single cloud facility. This dimensional shift from centralized to distributed architecture allows data processing to occur closer to the data source, reducing transmission latency and enabling simultaneous computation across multiple nodes.
2Productivity
If machine learning networks are implemented in cloud compute facilities with significant resources, then computational performance is improved, but power consumption increases due to data transfers
Solution Approach 1:
The patent segments the computation across multiple edge processing elements, allowing data to be processed locally without extensive transfers to centralized cloud facilities. This reduces the energy required for data transmission while distributing computational load, thereby reducing overall power consumption while maintaining performance.
Solution Approach 2:
The patent introduces edge devices as intermediary processing nodes between data sources and cloud facilities. These intermediaries perform local computation and filtering, reducing the volume of data that requires energy-intensive transmission to cloud facilities, thereby lowering overall power consumption.
3Productivity
If tensors are partitioned into slices and allocated to processing elements, then data transfer efficiency is improved, but system complexity increases
Solution Approach 1:
The patent segments tensors into smaller slices that can be independently processed by different processing elements. This segmentation improves data transfer efficiency by enabling parallel processing and reducing the bandwidth requirements for data transmission, while the modular nature of slicing keeps the system manageable despite increased complexity.
Solution Approach 2:
The patent implements dynamic allocation of tensor slices to processing elements based on available resources and computational requirements. This dynamic approach optimizes data transfer efficiency by adapting to changing conditions, while the flexibility of the system allows it to handle the increased complexity through adaptive resource management.
Data Source
AI summary
A system for compiling a machine learning network for execution on a plurality of interconnected processing elements performs the following. It selects splits (Ps,Qs,Ks) for individual layers of the machine learning network based on (a) costs of data transfer between processing elements assuming that inputs and outputs of individual layers have a same spatial split (Ps,Qs), and (b) costs of data transfer between processing elements resulting from differences in the spatial splits for inputs and outputs of individual layers. In addition, it selects the same splits (Ps,Qs,Ks) for layers that have the same spatial size P×Q. It then generates a computer program that allocates computations for executing the machine learning network to the processing elements according to the selected splits for the individual layers.