Heterogeneous SoC Compiler Allocating ML Inference Tasks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing embedded systems and edge devices face inefficiencies in executing machine learning inference due to the need for significant computational resources, leading to increased latency, network bandwidth, and costs, as they often rely on general-purpose processors rather than specialized chips optimized for machine learning.
Innovation Solution
A system-on-chip (SoC) with a heterogeneous compute environment that includes a machine learning accelerator (MLA) and a compiler, which allocates tasks across different processor cores, such as general-purpose, application-specific, and machine learning processors, optimizing for throughput, latency, and power consumption, while maintaining support for legacy code and enabling efficient implementation of machine learning networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If machine learning inference is implemented on general-purpose processors in embedded systems, then flexibility to perform various tasks is maintained, but computational efficiency and speed are significantly reduced
Solution Approach 1:
The processor is divided into separate functional units: a general-purpose processor for flexible task execution and a dedicated machine learning accelerator for computational efficiency. Each unit is optimized for its specific function, allowing the system to achieve both versatility and productivity.
Solution Approach 2:
The system combines multiple processor types (general-purpose and specialized ML accelerator) into a single heterogeneous computing platform. This universal architecture allows the same system to handle diverse workloads - from flexible task execution to optimized machine learning inference - without requiring separate dedicated hardware for each function.
2Productivity
If machine learning inference is implemented on dedicated machine learning chips, then computational speed and efficiency are optimized, but device complexity and cost increase
Solution Approach 1:
The machine learning accelerator is merged with the general-purpose processor into a single integrated chip architecture. This combination allows the system to achieve optimized computational speed for machine learning tasks while avoiding the complexity of entirely separate dedicated chips, as the ML accelerator shares infrastructure such as memory and interconnects with the general-purpose processor.
3Power
If machine learning inference is performed in cloud-based facilities, then significant computational resources are available, but latency and network bandwidth requirements increase
Solution Approach 1:
The machine learning accelerator extracts the computational workload from cloud-based facilities and implements it directly within the edge device. By taking out the ML inference function from the cloud and placing it locally in the embedded system, the solution eliminates network transmission delays while maintaining the computational power needed for effective machine learning inference.
Data Source
AI summary
A system-on-chip (SoC) integrated circuit product includes a machine learning accelerator (MLA). It also includes other processor cores, such as general purpose processors and application-specific processors. It also includes a network-on-chip for communication between the different modules. The SoC implements a heterogeneous compute environment because the processor cores are customized for different purposes and typically will use different instruction sets. Applications may use some or all of the functionalities offered by the processor cores, and the processor cores may be programmed into different pipelines to perform different tasks.