Configurable Function Approximation for Low-Latency Neural Quantization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing neural network computations face inefficiencies due to latency introduced by quantization, which degrades performance and lacks flexibility in trading off precision for speed.

Innovation Solution

A neural network processor with configurable arithmetic circuits that operate in different modes to adjust between precision and speed by using Taylor series expansion coefficients, allowing for inline quantization and flexible activation function approximation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If quantization is applied to improve efficiency and reduce complexity of neural network computation, then productivity is improved, but loss of time increases due to introduced latency

Engineering Contradiction:

Improvecomputation efficiencyVSAvoidprocessing latency

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The system dynamically switches between different computation modes (first mode with full precision arithmetic circuits, second mode with quantized arithmetic circuits) based on operational requirements. This dynamic adaptability allows the system to optimize between precision and speed in real-time, resolving the contradiction between computation efficiency and processing latency.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent changes the precision parameter of arithmetic operations by selecting between different computation modes. In the first mode, full precision parameters are used for accurate computation, while in the second mode, quantized parameters are used to improve efficiency. This parameter switching resolves the contradiction by allowing the system to adapt precision levels to specific computational needs.

Inventive Principle:

Principle #35Parameter changes

2Device complexity

If quantization is applied to reduce complexity, then device complexity is reduced, but measurement precision deteriorates in activation function approximation

Engineering Contradiction:

Improvecomputation complexityVSAvoidactivation function approximation precision

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The system dynamically adjusts the precision level of activation function approximation by switching between computation modes. When high precision is required, the first mode with full precision arithmetic circuits is used. When complexity reduction is prioritized, the second mode with quantized arithmetic circuits is used. This dynamic adjustment resolves the contradiction between device complexity and measurement precision.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent changes the precision parameter of arithmetic operations by selecting between different computation modes. In the first mode, full precision parameters are used for accurate activation function approximation. In the second mode, quantized parameters with fewer bits are used to reduce complexity. This parameter switching resolves the contradiction by allowing the system to adapt precision levels to specific computational needs.

Inventive Principle:

Principle #35Parameter changes

3Measurement precision

If full precision arithmetic circuits are used to maintain accuracy, then measurement precision is improved, but productivity decreases due to increased processing time

Engineering Contradiction:

Improvecomputation accuracyVSAvoidprocessing throughput

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The system dynamically switches between full precision and quantized computation modes based on operational requirements. When high accuracy is critical, the first mode with full precision arithmetic circuits is used. When processing throughput is prioritized, the second mode with quantized arithmetic circuits is used. This dynamic switching resolves the contradiction between measurement precision and productivity.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent changes the precision parameter of arithmetic operations by selecting between different computation modes. In the first mode, full precision parameters ensure high computation accuracy. In the second mode, quantized parameters with fewer bits improve processing throughput. This parameter switching resolves the contradiction by allowing the system to adapt precision levels to specific computational needs.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS12499353B1Configurable function approximation based on hardware selection of mapping table content

Publication Date: 2025.12.16 AMAZON TECH INC

AI summary

Systems and methods for performing hardware approximation of functions are provided. In one example, a system comprises a controller, a plurality of multiplexors, configurable arithmetic circuits, and a mapping table that stores a set of function parameters. According to a mode of operations, the controller may configure the plurality of multiplexors to forward the set of function parameters or a subset of the function parameters to the arithmetic circuits to compute an approximation result. In a case where the subset of the function parameters is forwarded to the arithmetic circuits, the controller may configure the arithmetic circuits to perform post-processing, such as quantization, of the approximation result.