Hybrid Neural Network With Intermittent Digital Weight Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural networks face a trade-off between accuracy and efficiency, as those using analog arrays excel in speed and power efficiency but suffer from lower accuracy due to quantization noise, while those with digital modules improve accuracy but falter in production environments with unpredictable data.
Innovation Solution
A neural network system that combines analog arrays with intermittently connected and activated digital modules, where a controller manages the activation based on accuracy and energy criteria to correct weights, ensuring a balance between accuracy and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If analog arrays are used in neural networks, then speed and power efficiency are improved, but accuracy deteriorates due to quantization noise
Solution Approach 1:
The patent combines analog arrays and digital modules into a hybrid neural network system. The analog arrays perform matrix-vector multiplications for fast computation, while digital modules correct quantization errors. This merging allows the system to achieve both high speed (from analog) and high accuracy (from digital correction), resolving the contradiction between speed and accuracy.
Solution Approach 2:
Digital modules serve as intermediaries that correct the quantization noise produced by analog arrays. The controller intermittently activates digital modules to clean up quantization errors in the analog computation results, allowing the analog system to maintain its speed advantage while achieving digital-level accuracy through the mediating correction process.
2Measurement precision
If digital modules are used in neural networks, then accuracy is improved, but power efficiency and speed deteriorate
Solution Approach 1:
The controller intermittently activates digital modules rather than keeping them continuously active. Digital modules are activated periodically to correct quantization errors when needed, and deactivated otherwise. This periodic action allows the system to achieve high accuracy when required while maintaining power efficiency by keeping digital modules dormant during periods when analog computation suffices.
Solution Approach 2:
Instead of using full digital computation for all operations, the system applies digital correction only partially - specifically for correcting quantization errors in analog computations. This partial use of digital modules provides sufficient accuracy improvement without the full power consumption cost of complete digital implementation.
3Measurement precision
If digital modules are continuously activated, then accuracy is maintained, but energy consumption increases
Solution Approach 1:
The controller implements periodic activation of digital modules based on accuracy criteria and energy criteria. Digital modules are activated only when accuracy requirements demand correction and energy availability permits, rather than continuous activation. This periodic scheduling maintains accuracy when needed while significantly reducing energy consumption compared to continuous operation.
Solution Approach 2:
The system dynamically adjusts the activation state of digital modules based on real-time conditions including accuracy requirements and energy availability. The controller monitors system state and adaptively switches digital modules on or off, making the system flexible and responsive rather than static. This dynamic control optimizes the trade-off between accuracy maintenance and energy conservation.
Data Source
AI summary
A neural network includes a plurality of analog arrays comprise all synaptic weights of the neural network. The neural network also includes digital modules that are co-trained along with the plurality of analog arrays. The digital modules are intermittently connected and intermittently activated when the neural network is in production. When activated and connected, the digital modules may correct weights of the analog arrays.