Arithmetic Processing Apparatus Dynamic Decimal Point Adjustment

Solution Overview

Problem

In arithmetic processing for deep learning, the initial stage of learning often experiences significant fluctuations in the Integer Word Length (IWL) of fixed-point numbers, leading to increased quantization errors and instability in the learning process due to gaps between estimated and actual decimal point positions, which can degrade recognition accuracy.

Innovation Solution

An arithmetic processing apparatus that dynamically adjusts the decimal point position of fixed-point number data based on statistical information obtained during training, using an offset to stabilize the IWL and reduce quantization errors, thereby maintaining high accuracy throughout the learning process.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Use of energy by moving object

If the decimal point position is determined based on statistical information during training, then the circuit scale and power consumption are reduced, but quantization errors increase and learning stability deteriorates in the initial stage

Engineering Contradiction:

Improvepower consumptionVSAvoidlearning stability

Core Design Contradiction:

Use of energy by moving objectVSReliability

Solution Approach 1:

The patent applies dynamics by making the decimal point position adjustable during the training process. The system transitions from a static fixed-point representation to a dynamic one where the decimal point position can be shifted based on the training stage. This allows the system to adapt to changing data distributions during training while maintaining the benefits of fixed-point arithmetic, thereby resolving the contradiction between power consumption reduction and learning stability.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent changes the parameter of decimal point position dynamically during training. By shifting the decimal point position according to the training stage and statistical information, the system optimizes the fixed-point representation to match the actual data distribution. This parameter change enables the system to maintain accuracy while using fixed-point numbers, resolving the contradiction between power efficiency and reliability.

Inventive Principle:

Principle #35Parameter changes

2Measurement precision

If the decimal point position is adjusted dynamically during training, then recognition accuracy is maintained, but the device complexity increases

Engineering Contradiction:

Improverecognition accuracyVSAvoidcontrol complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent implements feedback by using statistical information from the training process to determine the optimal decimal point position. The system continuously monitors the data distribution and adjusts the decimal point position accordingly, creating a closed-loop control mechanism. This feedback approach maintains recognition accuracy while keeping the control logic relatively simple, as it relies on straightforward statistical analysis rather than complex control algorithms.

Inventive Principle:

Principle #23Feedback

3Volume of moving object

If fixed-point numbers are used instead of floating-point numbers, then circuit scale is reduced, but quantization errors increase due to IWL fluctuations

Engineering Contradiction:

Improvecircuit scaleVSAvoidcalculation accuracy

Core Design Contradiction:

Volume of moving objectVSManufacturing precision

Solution Approach 1:

The patent applies dynamics by making the decimal point position adjustable during the training process. The system transitions from a static fixed-point representation to a dynamic one where the decimal point position can be shifted based on the training stage. This allows the system to adapt to changing data distributions during training while maintaining the benefits of fixed-point arithmetic, thereby resolving the contradiction between power consumption reduction and learning stability.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent changes the parameter of decimal point position dynamically during training. By shifting the decimal point position according to the training stage and statistical information, the system optimizes the fixed-point representation to match the actual data distribution. This parameter change enables the system to maintain accuracy while using fixed-point numbers, resolving the contradiction between power efficiency and reliability.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS11514320B2Arithmetic processing apparatus, control method, and non-transitory computer-readable recording medium having stored therein control program

Publication Date: 2022.11.29 FUJITSU LTD

AI summary

An arithmetic processing apparatus includes: a first determiner that determines, when a given learning model is repeatedly learned, an offset amount for correcting a decimal point position of fixed-point number data used in the learning in accordance with a degree of progress of the learning; and a second determiner that determines, based on the offset amount, the decimal point position of the fixed-point number data to be used in the learning. This configuration avoids lowering of the accuracy of a learning result of a learning model.