Multi-Chip Deep Learning Accelerator Package for Adaptive Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning accelerator (DLA) models lack post-manufacturing flexibility and efficiency, as they are not re-configurable, leading to excessive power consumption when higher computational capabilities are unnecessary, which is particularly problematic in power-constrained devices like IoT devices.
Innovation Solution
A DLA package with multiple DLA chips of varying computational capabilities allows for on-demand workload-aware compute deployment, where DLA models are assigned to chips based on their requirements, enabling scalable computational capability and reducing power usage by switching to less powerful chips when high accuracy is not needed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If DLA models are executed using high computational capability chips, then accuracy of results is improved, but power consumption increases
Solution Approach 1:
The system dynamically selects between multiple DLA chips with varying computational capabilities based on the confidence values of inference results. When confidence is sufficient, lower-power chips are used; when confidence is insufficient, higher-power chips are activated to improve accuracy, making the computational capability adaptive rather than static
Solution Approach 2:
The system changes the operational parameters by switching between different DLA chips with different computational capabilities (parameters) based on workload requirements. This allows the system to adjust its computational power parameter to match the actual needs, avoiding unnecessary power consumption while maintaining required accuracy
2Productivity
If DLA models are executed using high computational capability chips, then productivity is improved, but power consumption increases
Solution Approach 1:
The system dynamically adjusts computational resources by switching between chips with different capabilities based on actual inference needs. High-performance chips are activated only when necessary to meet productivity requirements, while lower-power chips handle routine tasks, creating a dynamic balance between speed and energy usage
Solution Approach 2:
The system applies partial computational action by using only the necessary computational capability required for each inference task. Instead of always deploying maximum computational power, the system uses just enough processing power to achieve sufficient confidence values, avoiding excessive energy consumption
3Adaptability or versatility
If multiple DLA chips with varying computational capabilities are deployed, then adaptability is improved, but device complexity increases
Solution Approach 1:
The system segments the computational workload across multiple DLA chips with different capabilities, assigning specific inference tasks to appropriate chips based on their computational strength. This segmentation allows the system to adapt to varying workload requirements while managing complexity through structured task distribution
Solution Approach 2:
The DLA package is designed as a universal computing unit that can handle multiple types of inference workloads by switching between chips with different capabilities. This multi-functionality allows a single package to adapt to various computational demands without requiring separate specialized systems for each workload type
Data Source
AI summary
A first deep learning accelerator (DLA) model can be executed using a first DLA chip of a DLA package. The first DLA chip can have a first computational capability and the first DLA model can have a first maximum accuracy value. Responsive to an accuracy value of first results from executing the first DLA model using the first DLA chip being less than a threshold accuracy value, signaling indicative of the first results can be provided directly to a second DLA chip of the DLA package and a second DLA model can be executed using the second DLA chip and the first results as inputs. The second DLA chip can have a second computational capability that is greater than the first computational capability. The second DLA model can have a second maximum accuracy value that is greater than the first maximum accuracy value.