Deep Learning Operation Data Type Selection for Lower Conversion Overhead

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing deep learning processors struggle to fully utilize the characteristics of tensor operations, where variations in tensor elements decrease as learning progresses, due to predetermined operation accuracy defined by hardware. This leads to inefficiencies and increased processing time, especially when converting between 32-bit floating-point and 8-bit or 16-bit fixed-point data types.

Innovation Solution

An information processing device that determines the optimal data type for each operation in a series of machine learning operations, modifies the data type to achieve higher accuracy than initially defined, and executes operations using the modified data type to minimize total processing time, including operation and conversion times.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If data type conversion is performed between 32-bit floating-point and 8-bit/16-bit fixed-point numbers to optimize operation speed, then operation time is reduced, but conversion delay is produced and total processing time is not guaranteed to be reduced

Engineering Contradiction:

Improveoperation speedVSAvoidconversion delay

Core Design Contradiction:

SpeedVSLoss of time

Solution Approach 1:

The system determines the optimal data type for each operation in advance before execution. By analyzing operation characteristics and tensor properties beforehand, the system pre-selects the appropriate data type (32-bit floating-point or 8-bit/16-bit fixed-point) to avoid unnecessary conversions during actual computation, thus eliminating conversion delay while maintaining optimized operation speed.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system dynamically adjusts data type selection based on operation characteristics and tensor properties rather than using a fixed data type throughout. This dynamic determination allows the system to adaptively choose between 32-bit floating-point and 8-bit/16-bit fixed-point numbers for each specific operation, optimizing the balance between conversion overhead and operation speed for each case.

Inventive Principle:

Principle #15Dynamics

2Productivity

If operation accuracy is lowered to improve power-to-operation performance, then processing time is reduced, but operation accuracy becomes insufficient for certain operations where tensor value variations increase

Engineering Contradiction:

Improvepower-to-operation performanceVSAvoidoperation accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The system applies different data type precision levels to different operations based on their specific characteristics. Instead of uniformly using low precision throughout, the system identifies operations where tensor value variations are small (suitable for 8-bit/16-bit fixed-point) versus operations where variations increase (requiring 32-bit floating-point), thereby applying appropriate precision locally to each operation type.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The system changes the data type parameter dynamically based on operation analysis. By monitoring and analyzing tensor value variations and operation characteristics, the system adjusts the precision parameter (selecting between 8-bit/16-bit fixed-point and 32-bit floating-point) for each operation to maintain sufficient accuracy while optimizing performance where possible.

Inventive Principle:

Principle #35Parameter changes

3Productivity

If data type is determined for each operation to shorten local processing time, then individual operation efficiency is improved, but total processing time for entire deep learning cannot be guaranteed to be reduced

Engineering Contradiction:

Improvelocal processing efficiencyVSAvoidtotal processing time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The system merges consecutive operations that can be performed with the same data type into unified processing units. By identifying sequences of operations that maintain consistent data type requirements, the system combines them to eliminate repeated conversion overhead, thereby reducing total processing time while maintaining local optimization benefits.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The system maintains data type consistency across consecutive operations to ensure continuous efficient processing. By determining the optimal data type for each operation and maintaining it where appropriate, the system ensures continuous useful action without interruption for conversions, thereby reducing total processing time while preserving local processing efficiency.

Inventive Principle:

Principle #20Continuity of useful action

Data Source

PatentUS12346779B2Information processing device, machine learning method, and non-transitory computer-readable storage medium for storing machine learning program of converting data type so that efficiency of deep learning is improved

Publication Date: 2025.07.01 FUJITSU LTD

AI summary

A method including: determining a data type to be used for each of a plurality of operations, the plurality of operations being a series of operations to be used in machine learning processing; reducing a total processing time by modifying the determined data type to be used in each of the operations to a data type that is more accurate than the data type defined by the determining, the total processing time being a time period including an operation time for each of the plurality of operations and a conversion time taken to convert the data type; and executing each of the plurality of operations by using the modified data type of each of the plurality of operations.