Lower-Precision Operand Series for Accurate Matrix Computation

Solution Overview

Problem

Current matrix calculations using lower-precision data formats for processors result in less precise outcomes despite increased speed, necessitating a method to perform high-accuracy calculations while maintaining optimal speed.

Innovation Solution

Simulating high-precision calculations by estimating lower precision operands and generating a finite series to represent an approximation of high-precision operands, followed by precision conversion techniques to generate a high-precision result.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If lower-precision data format is used for matrix calculations, then calculation speed is improved, but result precision deteriorates

Engineering Contradiction:

Improvecalculation speedVSAvoidresult precision

Core Design Contradiction:

SpeedVSMeasurement precision

Solution Approach 1:

The patent segments the calculation process into two distinct phases: a first phase using lower-precision data formats for bulk matrix calculations to achieve high speed, and a second phase using higher-precision data formats for final accumulation operations to ensure accuracy. This temporal and functional segmentation allows each phase to optimize for its specific purpose, resolving the contradiction between speed and precision.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent dynamically adjusts the precision level of data formats based on the computational stage. Lower-precision formats (e.g., FP16, BF16) are used during intensive matrix multiplication phases where speed is critical, while higher-precision formats (e.g., FP32, FP64) are activated during final accumulation phases where precision is paramount. This dynamic switching enables the system to adapt precision to computational needs.

Inventive Principle:

Principle #15Dynamics

2Measurement precision

If higher-precision data format is used for matrix calculations, then result precision is improved, but calculation speed deteriorates

Engineering Contradiction:

Improveresult precisionVSAvoidcalculation speed

Core Design Contradiction:

Measurement precisionVSSpeed

Solution Approach 1:

The patent segments the calculation process into two distinct phases: a first phase using lower-precision data formats for bulk matrix calculations to achieve high speed, and a second phase using higher-precision data formats for final accumulation operations to ensure accuracy. This temporal and functional segmentation allows each phase to optimize for its specific purpose, resolving the contradiction between speed and precision.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies higher-precision data formats selectively only to the extent necessary for final accumulation operations, rather than throughout the entire calculation process. By limiting high-precision computation to only the critical final stages where precision matters most, the system achieves the necessary accuracy without incurring the full speed penalty of using high-precision formats for all operations.

Inventive Principle:

Principle #16Partial or excessive action

3Productivity

If lower-precision format is used for bulk calculations, then productivity is improved, but manufacturing precision deteriorates

Engineering Contradiction:

Improvecalculation throughputVSAvoidcalculation accuracy

Core Design Contradiction:

ProductivityVSManufacturing precision

Solution Approach 1:

The patent segments the calculation process into two distinct phases: a first phase using lower-precision data formats for bulk matrix calculations to achieve high speed, and a second phase using higher-precision data formats for final accumulation operations to ensure accuracy. This temporal and functional segmentation allows each phase to optimize for its specific purpose, resolving the contradiction between speed and precision.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces an intermediary mechanism that manages data format conversion between lower-precision and higher-precision representations. This intermediary layer enables seamless transitions between precision levels, allowing bulk calculations to proceed efficiently in lower precision while ensuring final results achieve the required accuracy through controlled conversion to higher precision for accumulation operations.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS20260029989A1Lower precision operand representation

Publication Date: 2026.01.29 NVIDIA CORP

AI summary

Apparatuses, systems, and techniques to simulate high-precision calculations with a series expansion of lower precision tensor cores. In at least one embodiment, one or more multiplication operands of a first precision are represented by a sum of two or more operands of a different precision.