Neural Processing Tile Shuffling Nibbles to Skip Zero Weight Multiplications
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural processors face challenges in minimizing power consumption and latency, particularly in applications involving image or video processing from moving vehicles and battery-powered mobile devices.
Innovation Solution
A mixed-precision neural processor design that includes a tile configuration with weight registers, multipliers, and a shuffler, which processes nibbles of weights and activations efficiently, skipping unnecessary multiplications and leveraging sparsity to reduce power consumption and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If all nibbles are processed through multiplication operations, then processing completeness is ensured, but power consumption increases and latency increases due to unnecessary multiplications with zero weights
Solution Approach 1:
The patent extracts and removes unnecessary multiplication operations where weight nibbles are zero. The control logic detects zero weight nibbles and skips the corresponding multiplication and accumulation operations, extracting only the essential computational steps that contribute to the final result. This eliminates wasted energy on operations that would produce zero contributions anyway.
Solution Approach 2:
The patent implements skipping of unnecessary multiplication operations by detecting zero weight nibbles in advance. When a zero weight nibble is detected, the corresponding multiplication and accumulation steps are skipped entirely, allowing the processing to rush through the essential computations without being delayed by redundant operations.
2Loss of time
If all nibbles are processed through multiplication operations, then processing completeness is ensured, but latency increases due to unnecessary multiplications with zero weights
Solution Approach 1:
The patent extracts and removes unnecessary multiplication operations where weight nibbles are zero. The control logic detects zero weight nibbles and skips the corresponding multiplication and accumulation operations, extracting only the essential computational steps that contribute to the final result. This eliminates wasted time on operations that would produce zero contributions anyway.
Solution Approach 2:
The patent implements skipping of unnecessary multiplication operations by detecting zero weight nibbles in advance. When a zero weight nibble is detected, the corresponding multiplication and accumulation steps are skipped entirely, allowing the processing to rush through the essential computations without being delayed by redundant operations.
3Loss of energy
If sparsity exploitation is implemented with shufflers and selective multiplication, then power consumption is reduced and latency is reduced, but device complexity increases
Solution Approach 1:
The patent segments the weight data into individual nibbles and processes them separately through the shuffler. Each nibble is independently examined for zero values, and the shuffler reorders them to group non-zero nibbles together. This segmentation enables fine-grained sparsity exploitation at the nibble level, allowing selective multiplication only for non-zero segments.
Solution Approach 2:
The shuffler acts as an intermediary component between the weight register and the multiplication units. It receives weight nibbles, identifies zero values, and reorders the sequence to facilitate selective processing. This intermediary enables the control logic to efficiently skip zero multiplications while maintaining the computational pipeline.
Data Source
AI summary
A processor. In some embodiments, the processor includes: a first tile, the first tile being configured: to feed a first nibble from a third queue, through a first shuffler, to a first multiplier, and to multiply, in the first multiplier, the first nibble from the third queue by a first nibble of a third weight; to feed a second nibble from the third queue, through the first shuffler, to a second multiplier, and to multiply, in the second multiplier, the second nibble from the third queue by a second nibble of the third weight; and to feed a first nibble from a fourth queue, through the first shuffler, to a third multiplier, and to multiply, in the third multiplier, the first nibble from the fourth queue by a first nibble of a fourth weight.