Flexible Bit Truncation in ML Layers for Speed-Precision Balance

Solution Overview

Problem

Existing machine learning processing techniques face high computational complexity and data storage demands, particularly with complex AI and ML models, which can be misleading when data sources are corrupted, and require efficient methods to reduce complexity while maintaining accuracy.

Innovation Solution

Implementing flexible bit truncation in machine learning networks, where the number of bits truncated can be adjusted dynamically based on the application, reducing computational complexity and power consumption while maintaining processing accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If flexible bit truncation is applied to reduce computational complexity, then processing speed and power efficiency improve, but data precision and model accuracy may deteriorate

Engineering Contradiction:

Improveprocessing speedVSAvoiddata precision

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent implements dynamic bit truncation where the number of truncated bits is adjusted based on the specific application requirements. Different applications can have different truncation levels applied to different data types (weights, activations, gradients) during training and inference, allowing the system to optimize between speed and precision dynamically rather than using a fixed truncation level

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes the precision parameter of data representation by truncating a variable number of bits from floating-point numbers. This parameter change allows converting high-precision floating-point data to lower-precision formats (such as INT8, INT4, or custom formats) to reduce computational complexity while maintaining sufficient accuracy for the given application

Inventive Principle:

Principle #35Parameter changes

2Use of energy by moving object

If flexible bit truncation is applied to reduce power consumption, then energy efficiency improves, but computational accuracy may worsen

Engineering Contradiction:

Improvepower consumptionVSAvoidcomputational accuracy

Core Design Contradiction:

Use of energy by moving objectVSReliability

Solution Approach 1:

The system changes the precision parameter of data representation by truncating a variable number of bits from floating-point numbers. This parameter change allows converting high-precision floating-point data to lower-precision formats (such as INT8, INT4, or custom formats) to reduce computational complexity while maintaining sufficient accuracy for the given application

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent implements dynamic bit truncation where the number of truncated bits is adjusted based on the specific application requirements. Different applications can have different truncation levels applied to different data types (weights, activations, gradients) during training and inference, allowing the system to optimize between speed and precision dynamically rather than using a fixed truncation level

Inventive Principle:

Principle #15Dynamics

3Adaptability or versatility

If bit truncation settings are adjusted dynamically during runtime, then adaptability to different applications improves, but system complexity increases

Engineering Contradiction:

ImproveadaptabilityVSAvoidsystem complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent implements dynamic bit truncation where the number of truncated bits is adjusted based on the specific application requirements. Different applications can have different truncation levels applied to different data types (weights, activations, gradients) during training and inference, allowing the system to optimize between speed and precision dynamically rather than using a fixed truncation level

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system applies different bit truncation settings to different parts of the ML network (weights, activations, gradients) and different layers based on their specific requirements. This local quality approach allows critical layers to maintain higher precision while less critical layers use aggressive truncation, optimizing the balance between accuracy and efficiency at each location

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS20260037809A1Machine learning processing using flexible bit truncation

Publication Date: 2026.02.05 UNIV OF SOUTH ALABAMA

AI summary

A machine learning network is accessed. The network includes one or more processing layers. At least one of the processing layers is sourced with flexible bit truncation storage hardware. A flexible bit truncation setting is determined for the at least one of processing layers. The determining is based on an application to be executed on the network. At least one additional flexible bit truncation setting is determined, enabling at least two processing layers to be sourced with flexible bit truncation storage hardware. At least one of the additional flexible bit truncation settings is different from the flexible bit truncation setting. The flexible bit truncation setting is programmed in the flexible bit truncation storage hardware of the processing layers. The application is executed using the flexible bit truncation setting. The flexible bit truncation storage hardware comprises a static RAM (SRAM).