DSP Block Mixed-Precision Routing Without AI Retraining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning and artificial intelligence applications require significant compute power and retraining for sparsity, which is time-consuming and power-intensive, while structured mixed-precision operations can reorganize trained networks without retraining to achieve compute and power savings.
Innovation Solution
Implementing a DSP block in integrated circuit devices that perform structured mixed-precision operations by decomposing tensor columns into sub-columns with multi-level crossbar architectures and multiplexer patterns, allowing routing of inputs to multiple multipliers and cascading data across DSP blocks for efficient calculations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If sparsity is used to reduce compute needed for AI operations, then compute power is reduced, but retraining time increases and device power output increases
Solution Approach 1:
The patent applies parameter changes by transitioning from binary sparsity (0/1 masking) to structured mixed-precision operations that use multiple precision levels (e.g., FP16, FP32, INT8) for different weight elements. This allows the system to maintain computational efficiency while avoiding retraining, as the precision levels are selected based on weight magnitude distributions rather than requiring complete model retraining when sparsity is applied.
2Productivity
If sparsity is used to reduce compute needed for AI operations, then compute power is reduced, but device power output increases
Solution Approach 1:
The patent implements local quality by assigning different precision levels to different weight elements based on their importance and distribution characteristics. Critical weights with larger magnitudes use higher precision (FP32), while less critical weights use lower precision (FP16 or INT8). This localized precision assignment reduces overall computational energy consumption compared to uniform precision or sparsity-based approaches, as it avoids the need for high-power retraining operations while maintaining model accuracy.
3Productivity
If structured mixed-precision operations are implemented, then compute savings are achieved, but device complexity increases
Solution Approach 1:
The patent applies universality by designing a single DSP block architecture that can perform multiple precision operations (FP16, FP32, INT8) and various computational functions (matrix multiplication, convolution, pooling) using the same hardware resources. The multiplexer network and weight registers are configured to handle different precision levels and operation types, eliminating the need for separate dedicated hardware for each precision level or operation type, thus reducing overall device complexity despite the advanced functionality.
Data Source
AI summary
This disclosure is directed to a digital signal processing (DSP) block that includes multiple weight registers configurable to receive and store a first plurality of values having multiple precisions, and multiple multipliers that are each configurable to receive a respective value of the first plurality of values. The DSP block further includes one or more inputs configurable to receive a second plurality of values, and a multiplexer network configurable to receive the second plurality of values and route each respective value of the second plurality of values to a multiplier of the multipliers. The multipliers are configurable to simultaneously multiply each value of the first plurality of values by a respective value of the second plurality of values to generate a plurality of products. Additionally, the DSP block includes adder circuitry configurable to generate a first sum and a second sum based on the plurality of products.