Systolic Neural Network Engine Backpropagation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional computer processors are inefficient for neural network processing due to high computational costs and power consumption, especially during training, and they struggle with the wiring complexity of large neural networks.
Innovation Solution
A systolic neural network engine with a systolic array of data processing units (DPUs) that uses systolic pulses for layer-to-layer data transfer, enabling efficient processing of neural networks on a single chip and supporting both training and inference stages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional computer processors are used for neural network processing, then general-purpose computing is achieved, but processing time and power consumption increase significantly
Solution Approach 1:
The processor is divided into multiple processing elements (PEs) arranged in a systolic array architecture. Each PE handles specific neural network computations independently, enabling parallel processing of multiple data points simultaneously. This segmentation allows the system to achieve high throughput for matrix multiplications and convolution operations while maintaining lower power consumption per processing unit.
Solution Approach 2:
The patent transitions from conventional sequential or loosely-parallel processing to a two-dimensional systolic array architecture. Data and weights flow through the array in a wave-like pattern, with computations occurring at multiple stages simultaneously. This dimensional transformation enables O(n²) parallel operations to be performed in O(n) time steps, dramatically improving processing speed while keeping power consumption manageable through localized data movement.
2Adaptability or versatility
If conventional processors handle large neural networks, then computational flexibility is maintained, but wiring complexity increases
Solution Approach 1:
Each processing element in the systolic array is designed as a universal unit capable of performing multiple functions: multiplying inputs by weights, accumulating partial sums, applying activation functions, and passing results to neighboring PEs. This universality allows the same hardware structure to handle different neural network architectures (fully connected, convolutional, recurrent) without requiring complex reconfiguration of interconnections, thereby reducing wiring complexity while maintaining adaptability.
Solution Approach 2:
The systolic array employs dynamic data flow patterns where the direction and timing of data movement are controlled by clock signals and control logic. The same physical wiring infrastructure can support different computational patterns by changing the timing and routing control, enabling flexible adaptation to various neural network operations without physically rewiring the array.
3Adaptability or versatility
If backpropagation is implemented on conventional processors, then training capability is achieved, but computational cost and processing time increase
Solution Approach 1:
The systolic array maintains continuous data flow throughout the processing elements during both forward propagation and backpropagation. Gradient computations during backpropagation reuse the same data paths and processing elements used during forward propagation, eliminating idle periods and ensuring that all hardware resources remain actively engaged. This continuous operation significantly reduces the total time required for complete training iterations compared to conventional processors that require data reloading and repositioning.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of computer processing is disclosed comprising receiving a data packet at a processing node of a neural network, performing a calculation of the data packet at the processing node to create a processed data packet, attaching a tag to the processed data packet, transmitting the processed data packet from the processing node to a receiving node during a systolic pulse, receiving the processed data packet at the receiving node, performing a clockwise convolution on the processed data packet and a counter clockwise convolution on the processed data packet, performing an adding function and backpropagating results of the performed sigmoid function to each of the processing nodes that originally processed the data packet.