Systolic Processor Array Neural Network Architecture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systolic processor arrays used for deep convolutional neural networks (CNNs) are inefficient due to significant time spent organizing and regrouping intermediate data, leading to high resource demand and slow execution.
Innovation Solution
A neural network architecture with an array of processing elements, each equipped with local memory and data propagation circuitry for synchronous lateral mixing of neuron data, using an expanding spiral shift sequence to optimize data propagation and reduce latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing systolic processor arrays are used to implement CNNs, then neural network processing can be performed, but significant time is spent organizing and regrouping intermediate data, leading to slow execution
Solution Approach 1:
The patent applies preliminary action by pre-organizing the systolic processor array into a grid structure where processing elements are predetermined to handle specific data flow patterns. The local memories are pre-configured to store neuron data in a manner that eliminates the need for dynamic reorganization during processing, allowing neurons to be activated in a predetermined sequence without intermediate regrouping operations.
Solution Approach 2:
The patent segments the neural network processing into independent processing elements arranged in a grid, where each PE handles specific neuron computations. This segmentation allows parallel processing of different neuron groups simultaneously, eliminating the sequential bottlenecks that require data reorganization in traditional systolic arrays.
2Productivity
If existing systolic processor arrays are used to implement CNNs, then neural network computation can be performed, but processing resources are heavily demanded due to data reorganization operations
Solution Approach 1:
The patent creates a universal processing element design that can handle multiple neuron types (input, hidden, output layers) with the same basic computational structure. Each PE is multi-functional, capable of performing convolution, activation, and pooling operations through configurable parameters rather than dedicated hardware, reducing overall resource demand while maintaining high computational throughput.
Solution Approach 2:
The processing elements are designed to be self-sufficient with local memories that autonomously manage their own neuron data without requiring external control for data reorganization. Each PE independently retrieves necessary data from neighbors, performs computation, and stores results, eliminating the need for complex centralized data management resources.
3Productivity
If traditional data propagation methods are used in systolic arrays, then neural network layers can be processed sequentially, but latency is increased due to repeated data transfers
Solution Approach 1:
The patent implements a nested data structure where neuron data is organized in local memories that are hierarchically arranged. Each PE's local memory contains not only its own neuron data but also references to neighboring PEs' data, creating a nested access pattern that minimizes external transfers. This nesting allows neurons to access required data through localized memory operations rather than repeated physical data transfers across the array.
Data Source
AI summary
The present disclosure relates to an electronic circuit implementing a neural network, the electronic circuit comprising: an array of processing elements (PE) implementing one or more neurons of the neural network, each processing element comprising a data processing circuit, and a local memory configured to store neuron data; and data propagation circuitry configured to perform forward or reverse lateral mixing of the neuron data by propagating, synchronously by each processing element, the neuron data to the local memory of each processing element from the local memory of one or more neighboring processing elements, wherein each of the processing elements is configured to process, during a first processing iteration, the neuron data from the one or more neighboring processing elements in order to generate updated neuron data and to store the updated neuron data in the local memory for use during a subsequent processing iteration.