Mixed Precision Multiplexer for Deep Learning Acceleration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models, particularly convolutional neural networks (CNNs), require significant computational resources and time due to their large number of parameters and operations, which can be inefficient in terms of memory, processing power, and energy consumption, especially when higher precision is needed for accuracy.
Innovation Solution
Implementing mixed precision computing within CNNs, where operations can switch between high and low precision dynamically based on requirements, allowing for parallel processing and precision mode selection to optimize resource usage and speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high precision computing is used throughout the CNN, then accuracy is maintained, but computational resources and processing time increase significantly
Solution Approach 1:
The patent applies different precision levels to different parts of the CNN computation pipeline. Specifically, weight parameters are stored and processed in low precision (e.g., 8-bit integers) while activation values maintain higher precision (e.g., 32-bit floating point). This local differentiation allows the system to reduce computational burden in weight-heavy operations while preserving accuracy in activation-dependent computations, thereby improving processing speed without sacrificing overall model accuracy
Solution Approach 2:
The patent implements dynamic precision switching based on operational context. The system can adaptively select between low precision and high precision modes for different layers or operations within the CNN, depending on the specific computational requirements. This dynamic approach allows the system to optimize for speed when precision requirements are lower and switch to higher precision when accuracy is critical, resolving the contradiction between processing speed and computational precision
2Productivity
If low precision computing is used, then processing speed increases, but accuracy decreases
Solution Approach 1:
The patent applies different precision levels to different parts of the CNN computation pipeline. Specifically, weight parameters are stored and processed in low precision (e.g., 8-bit integers) while activation values maintain higher precision (e.g., 32-bit floating point). This local differentiation allows the system to reduce computational burden in weight-heavy operations while preserving accuracy in activation-dependent computations, thereby improving processing speed without sacrificing overall model accuracy
Solution Approach 2:
The patent applies low precision computing partially rather than universally. By restricting low precision usage to specific operations (primarily weight storage and multiplication) while maintaining high precision for other operations (activation functions, gradient computations), the system achieves speed improvements without the severe accuracy degradation that would result from applying low precision throughout the entire computation pipeline
3Measurement precision
If more computational resources are allocated, then accuracy is improved, but memory usage and energy consumption increase
Solution Approach 1:
The patent fundamentally changes the precision parameter of computational data representations. By storing weight parameters in low precision formats (reducing bit depth from 32-bit to 8-bit or lower), the system reduces memory bandwidth requirements and energy consumption for data transfer and storage. This parameter change allows the model to maintain acceptable accuracy while consuming significantly less energy, as lower precision operations require fewer computational cycles and less memory access
Data Source
AI summary
A device for deep learning acceleration with mixed precision may include a token generator configured to generate a token value and may include multiple multiplexers. Each multiplexer may include a load port configured to receive map data, a max pool port configured to receive max pool data, and matrix-matrix (MM) data input ports each configured to receive MM data based on MM output generated by an MM component. Each multiplexer may include an output port configured to provide output data to a single MM component. Each multiplexer may provide corresponding output data to a different MM component. Each multiplexer may be configured to select, based on the token value, an input from one of the load port, the max pool port, or a single MM data input port, of the MM data input ports, as the output data to be provided to the output port.