Dynamic Decimal Position Management in Tensor Processors

Solution Overview

Problem

Conventional matrix processor architectures require significant hardware resources to maintain precision in tensor computations, particularly in artificial neural networks, due to the use of floating-point number representations.

Innovation Solution

A computer system that dynamically manages the decimal position of tensor values based on collected statistics, adjusting the precision as needed to optimize hardware usage and prevent overflow or underflow during tensor computations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If floating-point number representations are used to maintain precision in tensor computations, then manufacturing precision is improved, but device complexity increases

Engineering Contradiction:

Improveprecision in tensor computationsVSAvoidhardware resources

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent implements dynamic precision management by adjusting the decimal position (precision) of fixed-point numbers based on the actual data distribution and computation requirements. Instead of using fixed floating-point representation throughout, the system dynamically determines the appropriate precision level for each tensor and computation operation, allowing precision to adapt to the specific needs of each layer and operation in the neural network.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes the precision parameter (decimal position) of fixed-point representations based on statistical analysis of tensor values. By collecting statistics about the range and distribution of values in tensors and adjusting the decimal position accordingly, the system optimizes precision for each specific computation while using simpler fixed-point arithmetic instead of floating-point operations.

Inventive Principle:

Principle #35Parameter changes

2Manufacturing precision

If floating-point number representations are used for storing tensor values, then manufacturing precision is improved, but loss of substance increases

Engineering Contradiction:

Improveprecision in tensor computationsVSAvoidhardware resources

Core Design Contradiction:

Manufacturing precisionVSLoss of substance

Solution Approach 1:

The patent dynamically adjusts the precision parameter (decimal position) of fixed-point representations based on statistical analysis of tensor values. By collecting statistics about the range and distribution of values in tensors and adjusting the decimal position accordingly, the system optimizes precision for each specific computation while using simpler fixed-point arithmetic instead of floating-point operations.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The system uses simpler fixed-point number representations instead of complex floating-point representations. By accepting that fixed-point arithmetic has limitations but compensating through dynamic precision adjustment and statistics-based optimization, the system achieves sufficient precision with less hardware resource consumption.

Inventive Principle:

Principle #27Cheap short-living objects (Disposable)

3Device complexity

If fixed precision is used in tensor computations, then device complexity is reduced, but manufacturing precision deteriorates

Engineering Contradiction:

Improvehardware resourcesVSAvoidprecision in tensor computations

Core Design Contradiction:

Device complexityVSManufacturing precision

Solution Approach 1:

The patent implements dynamic precision management by adjusting the decimal position (precision) of fixed-point numbers based on the actual data distribution and computation requirements. Instead of using fixed floating-point representation throughout, the system dynamically determines the appropriate precision level for each tensor and computation operation, allowing precision to adapt to the specific needs of each layer and operation in the neural network.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS10552119B2Dynamic management of numerical representation in a distributed matrix processor architecture

Publication Date: 2020.02.04 INTEL CORP

AI summary

A system receives and executes a sequence of tensor instructions, for example, instructions for performing a neural network computation. The system may be implemented as a multiprocessor architecture, for example, hardware for performing a neural network computation. A tensor instruction specifies a tensor computation receiving one or more input tensors for determining an output tensor. The system stores a decimal position associated with a plurality of values of a tensor. The system performs the tensor computation of a tensor instruction to determine a plurality of values of the output tensor. The system collects statistics describing the plurality of values of the output tensor and determines a decimal position for the plurality of values based on the collected statistics.