Ternary Weight Quantization for Programmable Integrated Circuit Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional Neural Networks (CNNs) and other learned parameter systems require extensive computations, leading to high energy, power, and memory usage due to redundancy in operations and weights, which is not efficiently addressed by existing technologies.
Innovation Solution
Implementing a programmable integrated circuit with reduced redundancy by using ternary weights and transposed weights, where redundant calculations are identified and combined, and zero values are ignored to optimize processing and memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional CNN implementations are used with full precision weights and operations, then computation accuracy is maintained, but energy consumption and processing time increase significantly
Solution Approach 1:
The patent applies parameter changes by transforming weight precision from conventional high-precision formats to ternary precision levels. Weights are quantized to three discrete values: -1, 0, and +1. This parameter transformation reduces the computational complexity of multiply-accumulate operations while maintaining sufficient accuracy for image recognition tasks, thereby reducing energy consumption without significantly compromising computation accuracy
Solution Approach 2:
The patent extracts and eliminates redundant computations from the CNN forward propagation process. By identifying that many weight values are zero or near-zero, the implementation removes these redundant operations entirely. The ternary quantization approach extracts only the essential computational elements needed for accurate classification, discarding unnecessary precision that contributes to energy consumption without adding value to the output
2Measurement precision
If conventional CNN implementations with full weight storage are used, then model accuracy is preserved, but memory usage and storage requirements increase
Solution Approach 1:
The patent changes the parameter representation of weights from high-precision floating-point or fixed-point formats to ternary quantized values. Each weight is represented by only 2 bits (encoding -1, 0, or +1), compared to 32 bits or more in conventional implementations. This parameter transformation reduces the storage memory required for weight matrices by more than an order of magnitude while preserving model accuracy through the sufficiency of ternary precision for capturing essential feature relationships
Solution Approach 2:
The patent uses copying techniques where identical weight values across different filters or layers are identified and stored only once, then referenced multiple times. The ternary quantization creates many repeated patterns of -1, 0, and +1 values that can be efficiently copied and reused across different computational operations, reducing overall memory requirements while maintaining the ability to perform accurate convolutions
3Reliability
If redundant operations are performed in CNN computations, then comprehensive feature processing is achieved, but processing time and computational overhead increase
Solution Approach 1:
The patent extracts and removes redundant operations from the CNN computation pipeline. By applying ternary quantization, operations involving zero weights are completely eliminated from the computational graph. The implementation identifies and removes redundant multiply-accumulate operations, replacing them with simpler conditional operations or direct memory access patterns, thereby reducing processing time while maintaining comprehensive feature processing through the preservation of non-zero weight operations
Solution Approach 2:
The patent segments the computational workload by separating zero-weight operations from non-zero-weight operations. The ternary quantization naturally divides the weight matrix into active (non-zero) and inactive (zero) components. This segmentation allows the implementation to process only the necessary non-zero elements, skipping redundant operations entirely, thus reducing processing time while maintaining feature processing completeness through focused computation on essential features
Data Source
AI summary
Systems and methods are included for efficiently implementing learned parameter systems (LPSs) on a programmable integrated circuit (PIC) via a computing engine. The computing engine receives an input set of learned parameters corresponding to use instances of an LPS. The computing engine reduces at least some redundancies and/or unnecessary operations using instance specific parameter values of the LPS, to generate a less redundant set of learned parameters and a corresponding less redundant LPS. The computing engine generates a netlist based on these, which may share computing resources of the PIC across multiple computations in accordance with the less redundant set of learned parameters and the corresponding less redundant LPS. The computing engine then programs the PIC with the netlist. That is, the netlist replaces use instances of at least some of the original learned parameters and its corresponding LPS and is executed instead of the original.