Mixed-Signal Matrix Vector Unit for Neural Network Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network processing technologies face inefficiencies in performing large-scale matrix operations due to the limitations of floating-point representation and the need for multiple configurations for different neural network models, which can lead to increased computational complexity and resource requirements.
Innovation Solution
The implementation of a processor with a matrix vector unit that converts digital values to analog values and performs operations in the analog domain, using a combination of floating-point and block-floating point representations to enhance dynamic range and efficiency, allowing for parallel processing of neural network operations across multiple nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If floating-point representation is used for neural network operations, then dynamic range is maintained, but computational complexity and resource requirements increase
Solution Approach 1:
The patent changes the numerical representation parameter from standard floating-point to block-floating-point format, where a single exponent is shared across multiple mantissas. This parameter change reduces the bits required per number while maintaining dynamic range, thereby reducing computational complexity without sacrificing adaptability
Solution Approach 2:
The patent creates a unified number format (block-floating-point) that can represent both very large and very small numbers within a single format, eliminating the need for multiple specialized formats. This universal format reduces the complexity of handling different data types while maintaining the dynamic range needed for neural network operations
2Productivity
If multiple configurations are used for different neural network models, then model-specific optimization is achieved, but resource requirements and setup complexity increase
Solution Approach 1:
The block-floating-point format serves as a universal representation that works across different neural network models and operations. Instead of configuring separate formats for different models, this single format adapts to various scenarios through its shared exponent mechanism, reducing configuration complexity while maintaining optimization capabilities
Solution Approach 2:
The patent introduces dynamic exponent sharing where the exponent applies to a block of numbers rather than individually. This dynamic structure allows the same configuration to efficiently handle varying data ranges across different models without requiring reconfiguration, achieving model-specific optimization through a universal dynamic format
3Measurement precision
If digital-domain operations are used, then precision is maintained, but processing speed and throughput are limited
Solution Approach 1:
The patent replaces traditional digital-domain mechanical computation with analog-domain electrical computation using current-mode circuits. Analog multipliers naturally perform multiplication through current relationships, providing high-speed parallel processing while maintaining precision through the block-floating-point representation that guides the analog computation
Data Source
AI summary
Processors and methods for neural network processing are provided. A method in a processor including a matrix vector unit is provided. The method includes receiving vector data and actuation vector data corresponding to at least one layer of a neural network model for processing using the matrix vector unit, where each of digital values corresponding to the vector data and the actuation vector data is represented in a sign magnitude format. The method further includes converting each of the digital values corresponding to at least one of the vector data or the actuation vector data to corresponding analog values and multiplying the vector data and the actuation vector data in an analog domain and providing corresponding multiplication results in a digital domain.