Neural Network Processor Dynamic Precision Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks face inefficiencies due to latency introduced by quantization, which degrades the overall performance and precision of computations, particularly in trading off between precision and speed of activation function processing.
Innovation Solution
A neural network processor with a controller and arithmetic circuits that operate in two modes: one for high precision using Taylor series expansion coefficients and another for lower precision with inline quantization, allowing a trade-off between precision and speed by configuring the arithmetic circuits to perform either a higher or lower degree Taylor series expansion and quantization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If quantization is applied to improve efficiency, then processing speed is improved, but precision is degraded
Solution Approach 1:
The patent implements dynamic switching between different computational modes (full precision and quantized) based on operational requirements. The system can adaptively select whether to apply quantization to specific neural network layers or operations, allowing flexibility in balancing precision and speed for different workloads and quality of service requirements.
Solution Approach 2:
The patent changes the precision parameter dynamically by switching between full precision arithmetic and quantized arithmetic. The system can adjust the quantization level and apply it selectively to different parts of the neural network computation, enabling parameter optimization based on performance requirements.
2Measurement precision
If higher degree Taylor series expansion is used, then precision is improved, but processing latency increases
Solution Approach 1:
The patent implements dynamic switching between different Taylor series expansion degrees (e.g., 2nd degree vs 3rd degree) based on precision requirements. The system can select the appropriate expansion degree for different neural network layers or operations, allowing optimization of the trade-off between precision and processing speed.
Solution Approach 2:
The patent changes the Taylor series expansion degree parameter to control the balance between precision and computational complexity. By adjusting this parameter, the system can achieve higher precision when needed while reducing computational overhead for less critical operations.
3Productivity
If inline quantization is applied, then processing speed is improved, but manufacturing precision is degraded
Solution Approach 1:
The patent implements dynamic control over when and where quantization is applied in the activation function pipeline. The system can switch between performing quantization inline (after each activation) or deferred (after multiple activations), allowing optimization of throughput versus precision based on workload characteristics.
Data Source
AI summary
Methods and systems for performing hardware approximation of function are provided. In one example, a system comprises a controller, configurable arithmetic circuits, and a mapping table. The mapping table stores a first set of function parameters in a first mode of operation and stores a second set of function parameters in a second mode of operation. Depending on the mode of operation, the controller may configure the arithmetic circuits to compute a first approximation result of a function at an input value based on the first set of function parameters, or to compute a second approximation result of the function at the input value based on the second set of function parameters and to perform post-processing, such as quantization, of the second approximation result.