Multiformat Controller for DNN Precision Adaptation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Deep Neural Networks (DNNs) face challenges in supporting multiple precision formats, leading to increased hardware costs, area, and performance penalties due to the need for various hardware compute engines, while conventional designs either incur high costs for supporting multiple precisions or suffer performance penalties by limiting precision support.

Innovation Solution

A multiformat controller that enables execution of both floating-point and integer operations across various precisions using a single hardware architecture, by decomposing floating-point operations into sign, exponent, and mantissa portions and mapping them onto integer-based compute engines, allowing for efficient data manipulation and reordering to perform multiply and accumulate operations without requiring specialized floating-point engines.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If multiple hardware compute engines are implemented to support various precision formats (float32, float16, int8, etc.), then the accelerator can support a wide range of DNN operations with different precision requirements, but the hardware area, cost, and complexity increase significantly

Engineering Contradiction:

Improveprecision supportVSAvoidhardware area

Core Design Contradiction:

Adaptability or versatilityVSArea of stationary object

Solution Approach 1:

The patent implements a universal integer-based compute engine that can perform multiple precision operations (int4, int8, int16, float8, bfloat16, fp16) through software configuration and data reordering rather than having separate hardware engines for each precision. The same physical hardware resources are reused across different precision formats, eliminating the need for dedicated floating-point hardware while maintaining support for various DNN operation precisions

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system dynamically changes operational parameters (precision format, data layout, compute engine configuration) through software control to adapt to different DNN workload requirements. By modifying data reordering strategies and compute engine parameters rather than hardware architecture, the system achieves precision adaptability without proportionally increasing hardware area

Inventive Principle:

Principle #35Parameter changes

2Area of stationary object

If only a few precision formats are supported in the hardware, then the hardware area and cost are reduced, but performance penalties occur and applicability is limited

Engineering Contradiction:

Improvehardware areaVSAvoidperformance

Core Design Contradiction:

Area of stationary objectVSProductivity

Solution Approach 1:

The patent replaces specialized floating-point hardware mechanics with integer-based compute engine mechanics enhanced by software data reordering. Instead of using dedicated floating-point ALUs, the system uses integer arithmetic units with software-managed data transformation, achieving comparable performance for supported precisions while reducing hardware area

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system performs preliminary data reordering and transformation operations before data reaches the compute engine. By pre-processing data into formats suitable for integer-based computation (reordering, packing, unpacking), the system enables efficient execution on simplified hardware without sacrificing the ability to handle various precision formats

Inventive Principle:

Principle #10Preliminary action

3Measurement precision

If specialized floating-point hardware engines are implemented to execute FP operations, then FP computation accuracy and performance are improved, but the hardware cost, area, and power consumption increase

Engineering Contradiction:

ImproveFP computation accuracyVSAvoidhardware resources

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The system creates a software-based copy of floating-point computation functionality using integer arithmetic. By implementing FP operations through sequences of integer operations with appropriate data reordering and transformation, the system replicates floating-point computation accuracy without requiring physical floating-point hardware components

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent introduces software data reordering and transformation logic as an intermediary between the integer-based compute engine and floating-point operations. This intermediary layer translates FP operations into equivalent integer operations, maintaining computational accuracy while eliminating the need for specialized FP hardware

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS20220382514A1Control logic for configurable and scalable multi-precision operation

Publication Date: 2022.12.01 INTEL CORP

AI summary

Systems, apparatuses, and methods include technology that determines whether an operation is a floating-point based computation or an integer-based computation. When the operation is the floating-point based computation, the technology generates a map of the operation to integer-based compute engines to control the integer-based compute engines to execute the floating-point based computation. When the operation is the integer-based computation, the technology controls the integer-based compute engines to execute the integer-based computation.