Low-Bit-Width Dot Product Engine for High-Precision Summation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current hardware solutions for artificial intelligence computations face challenges in being flexible and computationally efficient while performing complex operations like summations and dot products, especially when dealing with different bit widths and formats, which increases hardware complexity and expense.
Innovation Solution
A hardware system that includes a dot product processing component capable of calculating dot products of higher-bit-width numbers using lower-bit-width engines, with additional hardware to align and accumulate results, allowing for efficient processing of vectors across different bit widths, such as 8-bit integers and 32-bit integers or 16-bit floating-point numbers, by splitting and recombining data in multiple iterations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a low-bit-width dot product engine is used to process high-bit-width numbers, then hardware complexity and cost are reduced, but processing precision and accuracy deteriorate
Solution Approach 1:
The patent divides high-bit-width numbers into multiple segments that can be processed by the low-bit-width dot product engine. Each segment is processed separately and then recombined to reconstruct the final result, enabling the engine to handle numbers larger than its native bit width while maintaining accuracy.
Solution Approach 2:
The patent introduces additional processing dimensions by implementing multiple processing stages and intermediate accumulation registers. The computation is extended across multiple time steps and spatial locations, transforming a single-stage precision loss problem into a multi-stage reconstruction process that recovers the original precision.
2Adaptability or versatility
If additional hardware components are added to align and accumulate results, then processing capability for high-bit-width numbers is improved, but device complexity increases
Solution Approach 1:
The patent designs the additional hardware components to serve multiple functions: the alignment component not only aligns segments but also handles different data formats and bit widths; the accumulation registers serve both as temporary storage and as part of the computation pipeline. This multi-functionality reduces the need for separate dedicated components for each function.
Solution Approach 2:
The patent introduces intermediate data structures and temporary storage elements that act as mediators between the low-bit-width engine and the final high-bit-width result. These intermediaries facilitate the transformation and recombination of segmented data without requiring complex direct processing paths.
Data Source
AI summary
A device (e.g., an integrated circuit chip) includes a dot product processing component, a data alignment component, and an accumulator. The dot product processing component is configured to calculate a dot product of a first group of elements stored in a first storage unit with a second group of elements, wherein: each element of the first group of elements is represented using a first number of bits, each value of a group of values stored in the first storage unit is represented using a second number of bits greater than the first number of bits, and each value of the group of values is stored as split segments across more than one element of the elements of the first group of elements. The data alignment component is configured to receive results of the dot product processing component and modify one or more of the results of the dot product processing component. The accumulator is configured to sum outputs of the data alignment component to at least in part determine a sum of the group of values.