Unified Sparse Arithmetic Processor for Integer and Floating-Point Outliers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing processors for neural networks inefficiently utilize processing resources due to a significant fraction of relatively small numbers and a small fraction of large outliers, requiring separate hardware for integer and floating-point operations, which consumes chip area and power.
Innovation Solution
A processing circuit that performs both integer and floating-point operations by forming partial products using multipliers and adders, accommodating outlier values through offset additions and shifting, and rearranging weight nibbles for efficient use of hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If separate dedicated hardware is used for integer and floating-point operations, then operation accuracy is improved, but chip area and power consumption increase significantly
Solution Approach 1:
The patent implements a unified multiplier unit that can perform both integer and floating-point multiplication operations by dynamically selecting between different operational modes. The same hardware resources (multiplier, adder, register file) are reused for both operation types, eliminating the need for separate dedicated hardware for each operation type while maintaining the accuracy benefits of both integer and floating-point processing
Solution Approach 2:
The patent merges the integer and floating-point processing paths into a single integrated computational unit. The multiplier unit combines integer multiplication logic with floating-point multiplication logic, sharing common resources such as the adder unit, register file, and control logic. This consolidation reduces chip area while preserving the precision advantages of both operation types through proper data format handling
2Measurement precision
If separate dedicated hardware is used for integer and floating-point operations, then operation accuracy is improved, but power consumption increases significantly
Solution Approach 1:
The patent implements a unified multiplier unit that can perform both integer and floating-point multiplication operations by dynamically selecting between different operational modes. The same hardware resources (multiplier, adder, register file) are reused for both operation types, eliminating the need for separate dedicated hardware for each operation type while maintaining the accuracy benefits of both integer and floating-point processing
Solution Approach 2:
The patent merges the integer and floating-point processing paths into a single integrated computational unit. The multiplier unit combines integer multiplication logic with floating-point multiplication logic, sharing common resources such as the adder unit, register file, and control logic. This consolidation reduces chip area while preserving the precision advantages of both operation types through proper data format handling
3Productivity
If fine-grain sparse operations are used, then resource utilization is improved, but handling of outlier values becomes more complex
Solution Approach 1:
The patent applies different processing strategies to different parts of the data based on their characteristics. It identifies and separates outlier values from normal values, applying specialized handling logic only to outliers while using optimized fine-grain sparse processing for the majority of normal values. This localized approach maintains high resource utilization for common operations while providing targeted complexity management for edge cases
Solution Approach 2:
The patent extracts and isolates outlier values from the main data stream, processing them through separate dedicated logic paths. By removing outliers from the primary fine-grain sparse processing pipeline, the system maintains efficient resource utilization for the bulk of operations while handling exceptional cases through a separate, purpose-built mechanism that does not impact overall system performance
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A processor for fine-grain sparse integer and floating-point operations and method of operation thereof. In some embodiments, the method includes forming a first set of products and forming a second set of products. The forming of the first set of products may include: multiplying, in a first multiplier, a first activation value by a least significant sub-word and a most significant sub-word of a first weight form a first partial product and a second partial product; and adding the first partial product and the second partial product. The forming of the second set of products may include: multiplying, in the first multiplier, a second activation value by a first sub-word and a second sub-word of a mantissa to form a third partial product and a fourth partial product; and adding the third partial product and the fourth partial product.