Sparse Neural Processor for Mixed Integer and Floating-Point Multiply
Solution Overview
Problem
Existing processors for neural networks inefficiently utilize processing resources due to a mix of integer and floating-point operations, with a significant fraction of numbers being small and only a small fraction being large, leading to excessive chip area and power consumption.
Innovation Solution
A processing circuit that performs both integer and floating-point operations by using multipliers to handle activation values and weights, including least and most significant sub-words, and offset addition to efficiently process outlier values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If separate hardware is used for integer and floating-point operations, then operational accuracy is maintained, but chip area and power consumption increase significantly
Solution Approach 1:
The patent combines integer and floating-point multiplication capabilities into a single hardware unit. The multiplier is configured to accept different input formats (integer or floating-point) and perform the appropriate operation type, eliminating the need for separate dedicated hardware for each operation type while maintaining accuracy for both.
Solution Approach 2:
The multiplier unit is designed as a universal component that can perform both integer multiplication and floating-point multiplication. By making the hardware multi-functional, the patent reduces the total chip area required while preserving the precision needed for both operation types through format-aware processing logic.
2Measurement precision
If separate hardware is used for integer and floating-point operations, then operational accuracy is maintained, but power consumption increases significantly
Solution Approach 1:
The patent merges integer and floating-point operation capabilities into a single power-consuming unit. By consolidating the hardware, the patent eliminates redundant power consumption from having separate operational units, while maintaining accuracy through format-specific processing within the unified structure.
3Measurement precision
If all weights are processed with full precision, then computational accuracy is maintained, but processing time and computational resources increase
Solution Approach 1:
The patent segments weight processing by dividing weights into outlier weights (requiring full precision) and non-outlier weights (allowing reduced precision). This segmentation allows the system to process the majority of weights faster with lower precision while maintaining overall accuracy by carefully handling the critical outlier weights with full precision multiplication.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A processor for fine-grain sparse integer and floating-point operations and method of operation thereof. In some embodiments, the method includes forming a first set of products, and forming a second set of products. The forming of the first set of products may include: multiplying, in a first multiplier, a second multiplier, and a third multiplier, the first activation value by a first least significant sub-word, a second least significant sub-word, and a most significant sub-word; and adding a first resulting partial product and a second resulting partial product. The forming of the second set of products may include forming a first floating point product, the forming of the first floating point product including multiplying, in the first multiplier, a first sub-word of a mantissa of an activation value by a first sub-word of a mantissa of a weight, to form a third partial product.