Fused Partial-Product Addition With Wider Accumulators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in efficiently processing large data sets with small input terms while avoiding overflow or saturation during multiply accumulate operations, particularly in neural network calculations where the accumulator should tolerate twice the number of bits of the input terms to prevent overflow or saturation.
Innovation Solution
The implementation of a fused multiply accumulate (FMA) operation that uses different sized operands, where packed data elements of the same position from two sources are multiplied and the results are added to a corresponding full-sized packed data element from a third source, with the accumulator sizes being twice, four times, or eight times the size of the input sources, and includes saturation and rounding mechanisms to handle overflow.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the accumulator size is increased to prevent overflow during multiply accumulate operations, then the reliability of computation is improved, but the device complexity increases
Solution Approach 1:
The patent applies parameter changes by dynamically selecting accumulator sizes (32-bit, 64-bit, or 128-bit) based on the operand types and operation requirements. This allows the system to adjust the accumulator precision parameter to match the computational needs, preventing overflow while avoiding unnecessary complexity when lower precision suffices.
Solution Approach 2:
The system implements dynamic accumulator size selection where the accumulator width is not fixed but adapts based on the input operand sizes and operation characteristics. This dynamic adjustment enables the system to optimize between computational reliability and device complexity for each specific operation.
2Measurement precision
If larger accumulators are used to avoid saturation in neural network calculations, then the measurement precision is improved, but the use of energy increases
Solution Approach 1:
The patent changes the accumulator precision parameter dynamically based on the computational task requirements. For neural network operations, the system selects appropriate accumulator sizes (32-bit or 64-bit) based on the input data types and operation characteristics, ensuring sufficient precision to prevent saturation while minimizing energy consumption by avoiding unnecessarily large accumulators.
Solution Approach 2:
The system dynamically adjusts accumulator size according to the specific operation requirements, allowing the measurement precision to be optimized only when needed. This dynamic approach prevents continuous use of high-precision accumulators that would waste energy in cases where lower precision is sufficient.
3Reliability
If the accumulator size is doubled to prevent overflow, then the reliability is improved, but the manufacturing precision requirements increase
Solution Approach 1:
The patent implements parameter changes by selecting from discrete accumulator size options (32-bit, 64-bit, 128-bit) based on the operational requirements. This approach to manufacturing precision uses standard bit width increments that are well-defined in hardware architecture, avoiding the need for custom precision requirements while still achieving reliable overflow prevention through appropriate accumulator sizing.
Data Source
AI summary
Embodiments of systems, apparatuses, and methods for fused multiple add. In some embodiments, a decoder decodes a single instruction having an opcode, a destination field representing a destination operand, and fields for a first, second, and third packed data source operand, wherein packed data elements of the first and second packed data source operand are of a first, different size than a second size of packed data elements of the third packed data operand. Execution circuitry then executes the decoded single instruction to perform, for each packed data element position of the destination operand, a multiplication of a M N-sized packed data elements from the first and second packed data sources that correspond to a packed data element position of the third packed data source, add of results from these multiplications to a full-sized packed data element of a packed data element position of the third packed data source, and storage of the addition result in a packed data element position destination corresponding to the packed data element position of the third packed data source, wherein M is equal to the full-sized packed data element divided by N.