Neural Network Unit Output Buffer Feedback Masking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current artificial neural networks (ANNs) face challenges in achieving improved performance and efficiency in computations, particularly due to the need for efficient processing of large numbers of connections and weights, which can lead to underutilization of resources and increased computational time.
Innovation Solution
The processor design incorporates a neural network unit (NNU) with dynamically configurable neural processing units (NPUs) and a unique architecture that allows for efficient multiply-accumulate operations and activation functions, utilizing a rotater mechanism to process data and weights in a pipelined fashion, enabling high performance in neural network computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional neural network processing architectures are used, then computational accuracy is maintained, but computational time increases and resource utilization decreases
Solution Approach 1:
The neural network processing is divided into distinct pipeline stages including weight loading, data loading, multiply-accumulate operations, and output generation. Each stage operates independently and concurrently, allowing parallel processing of multiple neural network computations simultaneously, thereby reducing overall computational time while maintaining accuracy
Solution Approach 2:
Weights and data are pre-loaded into dedicated buffers before computation begins. The weight buffer and data buffer are populated in advance, allowing the computational units to immediately begin multiply-accumulate operations without waiting for memory access during computation, thus eliminating idle time and improving computational speed
2Productivity
If more neural processing units are added to handle large numbers of connections, then computational capacity increases, but resource underutilization occurs
Solution Approach 1:
The neural processing units are designed with universal functionality to handle various neural network operations including different activation functions (sigmoid, tanh, ReLU), multiple connection types, and diverse data formats. This multi-functionality allows a fixed number of NPUs to efficiently process varying computational loads without resource underutilization, as each NPU can adapt to different task requirements
Solution Approach 2:
The pipeline architecture ensures continuous utilization of neural processing units by maintaining a steady stream of computations through multiple pipeline stages. While one NPU is performing multiply-accumulate operations, another is loading weights, and a third is generating outputs, eliminating idle cycles and ensuring continuous productive action across all resources
3Productivity
If sequential processing methods are used, then implementation simplicity is maintained, but computational efficiency decreases
Solution Approach 1:
The processor employs dynamic pipeline control mechanisms that automatically adjust the flow of data and computations through different stages based on readiness signals. Control logic dynamically coordinates between weight loading, data processing, and output generation stages, enabling efficient parallel execution without requiring complex manual scheduling, thus improving computational efficiency while managing complexity through automated control
Data Source
Figure 1
Figure 2~3
Figure 4~6A
AI summary
An output buffer holds N words arranged as N/J mutually exclusive output buffer word groups (OBWG) of J words each. N processing units (PU) are arranged as N/J mutually exclusive PU groups each having an associated OBWG. Each PU has an accumulator, an arithmetic unit, and first and second multiplexed registers each having at least J+1 inputs and an output. A first input receives a memory operand and the other J inputs receive the J words of the associated OBWG. Each accumulator provides its output to a respective output buffer word. Each arithmetic unit performs an operation on the first and second multiplexed register outputs and the accumulator output to generate a result for accumulation into the accumulator. A mask input to the output buffer controls which words, if any, of the N words retain their current value or are updated with their respective accumulator output.