Segmentable Array Width Rotator for Neural Network Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network processors face inefficiencies in performing computations for deep learning applications, particularly in handling large datasets and weight sizes, due to limitations in data and weight size computation capabilities and the need for improved performance and efficiency in operations like convolutions and recurrent neural network calculations.
Innovation Solution
A processor architecture that includes a neural network unit (NNU) with multiple neural processing units (NPUs) capable of dynamically configuring to process various data sizes and performing operations like collective shifts, convolutions, and recurrent neural network calculations efficiently, utilizing shared activation function units and output buffer feedback mechanisms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the array width of neural processing units is fixed, then the hardware structure is simple, but the adaptability to different data sizes and weight sizes is limited
Solution Approach 1:
The patent implements a segmentable array width rotator that can dynamically reconfigure the array width of neural processing units based on the input data size and weight size. The rotator divides the NPU array into multiple segments and adjusts the active segment width to match the computation requirements, enabling the system to adapt from processing small datasets to large datasets without fixed hardware constraints.
2Speed
If separate weight memories are allocated for each rotator segment, then weight access is fast, but the memory structure becomes complex and resource utilization is low
Solution Approach 1:
The patent employs a single shared weight memory that serves all rotator segments simultaneously. The weight memory is designed with re-shapeable capabilities that allow it to dynamically adjust its output format to match the segment width requirements of any active rotator segment. This universal weight memory eliminates the need for multiple separate memory structures while maintaining fast weight access through direct addressing and flexible data formatting.
3Productivity
If the neural network unit processes large datasets with diverse sizes, then the computation capability is enhanced, but the number of clock cycles required increases
Solution Approach 1:
The segmentable array width rotator dynamically adjusts the active segment width to precisely match the data size being processed. When processing large datasets, the rotator activates more segments and increases the array width to process more elements in parallel, thereby reducing the number of clock cycles required. This dynamic reconfiguration allows the system to optimize its throughput for different dataset sizes without wasting clock cycles on underutilized hardware resources.
4Adaptability or versatility
If common weights are provided to multiple rotator segments, then resource utilization is improved, but the weight memory must be re-shapeable adding complexity
Solution Approach 1:
The weight memory implements re-shapeable functionality by dynamically changing its output parameter (segment width) based on the requirements of active rotator segments. The weight memory can reshape its data output to match different segment widths through configurable readout patterns and data formatting logic. This parameter change capability allows common weights to be efficiently shared across multiple segments with different width requirements, improving resource utilization while maintaining compatibility with diverse computational needs.
Data Source
AI summary
First/second memories hold rows of N weight/data words. The first memory address has log2 W bits and an extra bit. Each of N processing units (PU) of index J has first and second registers, an accumulator, an arithmetic unit performs an operation thereon to accumulate a result, first multiplexing logic for PUs 0 through (N/2)−1 receives first memory weight words J and J+(N/2) and for PUs N/2 through N−1 receives first memory weight words J and J−(N/2) and outputs a selected weight word to the first register, and second multiplexing logic receives second memory data word J and data word output by the second register of PU J−1 and outputs a selected data word to the second register. PU 0 second multiplexing logic also receives PU (N/2)−1 second register data word, and PU N/2 second multiplexing logic also receives PU N−1 second register data word.