Rotary Positional Encoding on 8-Bit Hardware With Mixed Precision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware components, such as ASICs, are often designed for lower bit-width data paths, leading to challenges in processing higher resolution data, resulting in underutilization of hardware resources and increased power consumption, while transitioning to newer models with higher precision requirements.
Innovation Solution
Implementing a mixed-precision pipeline using Rotary Positional Embedding (RoPE) to encode position information, where lower precision operations are performed in a fixed-bit domain and higher precision operations in a logic execution block, mitigating quantization errors through logarithmic transformations and exponentiation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If higher precision data processing is implemented, then measurement precision is improved, but use of energy increases and device complexity increases
Solution Approach 1:
The patent segments the computational pipeline into two distinct parts: a low-precision MAC array for bulk multiplication operations and a high-precision logic execution block for critical operations. This segmentation allows the system to achieve high precision position encoding only where necessary while keeping the majority of computations at lower precision, thereby reducing overall power consumption while maintaining measurement precision.
Solution Approach 2:
The patent applies local quality by using high-precision computation only in specific locations within the data flow - specifically in the logic execution block for operations involving logarithmic transformations and exponentiation. The majority of the system uses low-precision MAC operations. This localized application of high precision where most needed reduces energy usage while maintaining the required measurement precision for position encoding.
2Measurement precision
If higher precision data processing is implemented, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent divides the processing system into segmented functional blocks: MAC arrays for low-precision multiplication and separate logic execution blocks for high-precision operations. This segmentation allows each block to be optimized independently, reducing overall device complexity compared to a fully high-precision system while maintaining the required measurement precision through coordinated operation of the segments.
Solution Approach 2:
The patent introduces an intermediary logarithmic transformation that converts high-precision multiplication operations into operations that can be performed in a mixed-precision environment. This intermediary transformation allows the system to achieve high precision results without requiring the entire computational path to operate at high precision, thereby reducing device complexity.
3Use of energy by moving object
If lower bit-width hardware is used, then use of energy is reduced, but measurement precision deteriorates
Solution Approach 1:
The patent changes the parameter representation by using logarithmic transformations of the position encoding values. This parameter change allows the system to work with lower bit-width hardware while maintaining the effective precision required for accurate position encoding, as the logarithmic domain compresses the dynamic range and reduces the bits needed to represent values with sufficient precision.
Solution Approach 2:
The patent uses logarithmic transformation as an intermediary that bridges the gap between low-precision hardware and high-precision requirements. By transforming the problem into the logarithmic domain, the system can perform computations with lower precision hardware and then transform back to achieve the required measurement precision for position encoding.
4Adaptability or versatility
If mixed-precision pipeline is implemented, then adaptability is improved, but device complexity increases
Solution Approach 1:
The patent designs the MAC array and logic execution blocks to perform multiple functions: the MAC array handles both low-precision multiplication and can feed into different logic execution blocks, while the logic blocks handle logarithmic transformations, exponentiation, and other operations. This multi-functionality allows a single mixed-precision pipeline to adapt to different position encoding models and requirements without requiring separate dedicated hardware for each function, making the increased complexity worthwhile for the adaptability gained.
Data Source
AI summary
Embodiments include systems and methods for rotary positional embedding in a mixed-precision pipeline. An example method can be performed in a mixed-precision pipeline including a multiplier-accumulator (MAC) for a first bit width to execute a multiplication function and a logic execution block for a second, greater, bit width. The method includes operations executed by the circuit including obtaining, via a circuit for a first bit-width, an input tensor and a logarithm of an angle, θ; generating, by a multiplication function for inputs having the first bit-width, a product of a first element of the input tensor and a first element of the logarithm of θ, each of the first elements having the first bit-width. The method includes operations executed by the logic execution block including generating an exponent of the product to determine θ according to the second bit-width and generating a rotation matrix according to trigonometric functions of θ.