MAC Pipeline Weight Conversion Using Gaussian Floating Point
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing systems for image filtering face inefficiencies due to high memory allocation and bandwidth usage by filter weights stored in low precision formats, which affects the range and precision required for filtering operations.
Innovation Solution
The conversion of filter coefficients from initial data formats (e.g., fixed point or floating point) to Gaussian floating point data format, allowing for a symmetrical distribution that reduces memory allocation while maintaining suitable range and precision, and subsequent conversion to formats compatible with multiplier-accumulator circuitry for processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If filter weights are stored in low precision formats, then memory allocation is reduced, but range and precision required for filtering operations deteriorate
Solution Approach 1:
The patent changes the numerical data format parameter from traditional fixed-point or floating-point to Gaussian floating-point format. This parameter change allows filter weights to be represented with fewer bits while maintaining the necessary precision for filtering operations, as the Gaussian distribution naturally concentrates values around the mean, enabling efficient quantization to 8-bit or lower precision without significant loss of accuracy.
Solution Approach 2:
The patent introduces dynamic precision adaptation by using Gaussian floating-point format which can dynamically adjust the distribution of precision across different weight values. Weights with larger magnitudes receive more precision bits while smaller weights use fewer bits, optimizing the overall precision-memory tradeoff for the filtering operation.
2Loss of energy
If filter weights are stored in low precision formats, then memory bandwidth usage is reduced, but range and precision required for filtering operations deteriorate
Solution Approach 1:
The patent changes the data format parameter to Gaussian floating-point, which enables compact representation of filter weights with fewer bits. This reduces the volume of data that needs to be transferred between memory and processing units, thereby reducing memory bandwidth usage and energy consumption while preserving the essential precision characteristics through the Gaussian distribution properties.
3Quantity of substance
If filter weights are converted to Gaussian floating point data format, then memory allocation is reduced, but device complexity increases due to format conversion circuitry
Solution Approach 1:
The patent applies preliminary action by performing the Gaussian floating-point conversion during the weight initialization or loading phase, rather than during every filtering operation. The converted weights are then stored in the Gaussian format for subsequent use, avoiding repeated conversion operations and reducing the overall computational burden despite the initial conversion complexity.
Solution Approach 2:
The patent introduces format conversion circuitry as an intermediary component that bridges the traditional data format and the Gaussian floating-point format. This dedicated conversion module handles the format transformation efficiently, isolating the complexity to a specific component while allowing the rest of the filtering pipeline to operate with the optimized Gaussian format.
4Measurement precision
If filter weights use symmetrical distribution in Gaussian floating point format, then precision is improved, but device complexity increases due to conversion requirements
Solution Approach 1:
The patent changes the distribution parameter of filter weights from uniform or asymmetric distributions to a symmetrical Gaussian distribution. This parameter change ensures that weights are concentrated around the mean with predictable decay, enabling more efficient quantization and precision representation while reducing the dynamic range requirements and improving numerical stability in the filtering operations.
Data Source
AI summary
A system and/or an integrated circuit including a multiplier-accumulator execution pipeline which includes a plurality of MACs to implement a plurality of multiply and accumulate operations. A first memory stores filter weights having a Gaussian floating point (“GFP”) data format and a first bit length. A data format conversion circuitry includes circuitry to convert the filter weights from the GFP data format and the first bit length to filter weights having the data format and bit length that are different from the GFP data format and the first bit length. The converted filter weights are output to the MACs, wherein in operation, the MACs are configured to perform the plurality of multiply operations using (a) the input data and (b) the filter weights having the data format and bit length that are different from the GFP data format and the first bit length, respectively.