Dual Dot Product Engine Neuron for Edge AI
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
There is a need for artificial neural networks to reduce power consumption and increase flexibility in edge devices for local inference and on-device learning, as traditional centralized training and inference methods are inefficient for always-on applications and require improved processing capabilities.
Innovation Solution
The development of a neuron for artificial neural networks that incorporates dual dot product engines with different quantization levels and operational modes, allowing for selective enablement and disablement to optimize power consumption and accuracy, along with a combiner to combine outputs and apply activation functions, enabling efficient processing in edge devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If dual dot product engines with different quantization levels are used, then processing accuracy is improved, but device complexity increases
Solution Approach 1:
The weight set is segmented into two distinct sets: first weights with lower quantization levels and second weights with higher quantization levels. Each dot product engine processes one segment, allowing the system to achieve high accuracy for critical computations while maintaining low complexity for less critical ones. This segmentation resolves the contradiction by distributing the accuracy requirement across multiple specialized components rather than uniformly increasing complexity throughout the system.
Solution Approach 2:
Different quantization levels are assigned to different weight sets based on their specific computational requirements. The first dot product engine uses lower precision weights for computations where high accuracy is less critical, while the second engine uses higher precision weights where accuracy is paramount. This local differentiation of quality allows the system to optimize accuracy where needed without unnecessarily increasing complexity elsewhere.
2Use of energy by moving object
If the second dot product engine is selectively enabled, then power consumption is reduced, but processing capability is limited
Solution Approach 1:
The system dynamically adjusts its operational configuration by selectively enabling or disabling the second dot product engine based on real-time requirements. When processing accuracy requirements are moderate, only the first engine operates, reducing power consumption. When high accuracy is required, the second engine is enabled to supplement processing capability. This dynamic adaptability resolves the contradiction by allowing the system to optimize between power consumption and processing capability based on actual needs rather than maintaining a fixed configuration.
Solution Approach 2:
The first dot product engine is designed to handle a universal range of computational tasks independently, while the second engine provides enhanced capability for specific tasks requiring higher precision. The selective enablement mechanism allows the system to use the simpler first engine for most operations, reserving the more capable second engine only when necessary, thus achieving multi-functionality without always incurring the full complexity and power cost.
3Loss of time
If local inference is implemented in edge devices, then response time is improved, but power consumption increases
Solution Approach 1:
The inference processing is segmented into two stages corresponding to the two dot product engines. The first engine handles the majority of inference operations with lower power consumption, providing timely responses for common cases. The second engine is invoked only when the first engine's output requires refinement or when accuracy thresholds are not met, thus minimizing additional power consumption while maintaining fast response times for most operations.
Solution Approach 2:
The system performs partial inference using only the first dot product engine for many operations, accepting that some precision may be sacrificed for speed and power efficiency. When full accuracy is required, the second engine is activated to complete the inference process. This partial action approach allows local inference to provide improved response times compared to centralized processing, while keeping power consumption increases manageable by not always deploying full computational resources.
Data Source
AI summary
The present disclosure relates to a neuron for an artificial neural network, the neuron comprising: a first dot product engine and a second dot product engine. The first dot product engine is operative to: receive a first set of weights; receive a set of inputs; and calculate the dot product of the set of inputs and the first set of weights to generate a first dot product engine output. The second dot product engine is operative to: receive a second set of weights; receive the set of inputs; and calculate the dot product of the set of inputs and the second set of weights to generate a second dot product engine output. The neuron further comprises a combiner operative to combine the first dot product engine output and the second dot product engine output to generate a combined output, and an activation function module arranged to apply an activation function to the combined output to generate a neuron output.