Neural Network Quantization with Per-Region Decimal Point Control

Solution Overview

Problem

Existing CNN inference processing using fixed-point arithmetic faces challenges in maintaining inference accuracy, particularly in low-frame-rate videos due to low frame correlation, latency in decimal point position adjustment, and inability to adaptively control decimal point positions in response to biases in feature maps.

Innovation Solution

An arithmetic processing device and method that dynamically adjusts decimal point positions within feature maps by dividing them into units, analyzing each unit's operation result, determining optimal decimal point positions, and performing quantization to optimize fixed-point data precision.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If the decimal point position is controlled for each image or layer, then the inference accuracy can be improved, but the latency increases and the device complexity increases

Engineering Contradiction:

Improveinference accuracyVSAvoidlatency

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The feature map is divided into multiple division units, and the decimal point position is determined for each division unit independently. This segmentation allows parallel processing of different regions, reducing the overall latency while maintaining high precision for each local region.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The analysis unit performs analysis on the arithmetic operation result to determine the decimal point position in advance for each division unit. This preliminary determination of decimal point positions allows subsequent quantization operations to proceed efficiently without waiting for global analysis completion, thereby reducing latency.

Inventive Principle:

Principle #10Preliminary action

2Measurement precision

If the decimal point position is controlled for each image or layer, then the inference accuracy can be improved, but the device complexity increases

Engineering Contradiction:

Improveinference accuracyVSAvoiddevice complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The feature map is divided into multiple division units, and the decimal point position is determined for each division unit independently. This segmentation allows parallel processing of different regions, reducing the overall latency while maintaining high precision for each local region.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The analysis unit performs analysis on the arithmetic operation result to determine the decimal point position in advance for each division unit. This preliminary determination of decimal point positions allows subsequent quantization operations to proceed efficiently without waiting for global analysis completion, thereby reducing latency.

Inventive Principle:

Principle #10Preliminary action

3Device complexity

If a single decimal point position is used for the entire feature map, then the device complexity is reduced, but the arithmetic accuracy deteriorates locally where bias occurs

Engineering Contradiction:

Improvedevice complexityVSAvoidarithmetic accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

Instead of using a uniform decimal point position for the entire feature map, the patent determines different decimal point positions for different division units based on their local characteristics. This local optimization ensures that each region uses the most appropriate precision level, preventing local accuracy deterioration while keeping the overall system manageable.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS20260023958A1Arithmetic processing device, arithmetic processing methods, and arithmetic processing program

Publication Date: 2026.01.22 NT T INC

AI summary

An arithmetic processing device includes: an arithmetic unit configured to execute an arithmetic operation corresponding to each of layers constituting a neural network and output an arithmetic operation result; an analysis unit configured to perform, for each of division units obtained by dividing the arithmetic operation result by one or more units, an analysis according to the arithmetic operation result belonging to the division unit, and output an analysis result for each division unit; a decimal point position determination unit configured to determine a decimal point position indicating a dynamic range for each division unit on the basis of the analysis result for each division unit output by the analysis unit; and a quantization unit configured to perform quantization on the arithmetic operation result to become fixed-point data having a decimal point position determined for the division unit to which the arithmetic operation result belongs.