Systolic Array Neural Network Processor Parallel Matrix Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for computing convolutional neural network inferences are inefficient due to limited parallelization of calculations in software processors, which results in time-intensive matrix multiplications.
Innovation Solution
A special-purpose hardware circuit using a two-dimensional systolic array processes neural network inferences by sending kernel and activation input matrices to distinct cells, allowing for parallelized matrix multiplications and efficient computation of layer outputs through normalization and pooling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If software processors are used to compute convolutional neural network inferences, then the system can process neural network layers with flexible programming, but the calculation is time-intensive due to limited parallelization
Solution Approach 1:
The patent replaces the mechanical/software-based processing system with a hardware-based systolic array processor. The systolic array is a specialized hardware circuit designed specifically for matrix multiplication operations, which are fundamental to convolutional neural network computations. This substitution of general-purpose software processing with dedicated hardware processing resolves the contradiction by providing both the computational speed of hardware parallelization and the flexibility needed for neural network operations through configurable weight and activation input matrices.
2Ease of operation
If brute force matrix multiplication is used in software, then the processor can compute individual sums and products for convolution calculations, but the process is compute-intensive and time-intensive
Solution Approach 1:
The patent segments the matrix multiplication computation into multiple parallel processing units within the systolic array. Each processing element in the array simultaneously performs individual multiplications and accumulations on different elements of the matrices. This segmentation of the computation into parallel independent operations that can be executed simultaneously resolves the contradiction by maintaining computational simplicity while dramatically increasing processing throughput.
Solution Approach 2:
The patent transitions from sequential one-dimensional processing in software to two-dimensional parallel processing in the systolic array hardware. The array is organized as a grid of processing elements that operate in parallel across both spatial dimensions, enabling simultaneous computation of multiple matrix elements. This dimensional expansion from sequential to parallel processing resolves the contradiction between operational simplicity and processing speed.
3Manufacturing precision
If a large number of matrix multiplications are performed in large dimensional space, then complete convolution calculations can be achieved, but the processor architecture limits the degree of parallelization
Solution Approach 1:
The patent designs the systolic array processing elements to be universal and multi-functional, where each processing element can handle different weight inputs, activation inputs, and perform both multiplication and accumulation operations. This universality allows the same hardware structure to efficiently process matrices of various dimensions while maintaining full parallelization capability, resolving the contradiction between calculation accuracy and parallelization capability.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for computing a layer output for a convolutional neural network layer, the method comprising: receiving the layer input, the layer input comprising a plurality of activation inputs, the plurality of activation inputs represented as a multi-dimensional matrix comprising a plurality of depth levels, each depth level being a respective matrix of distinct activation inputs from the plurality of activation inputs; sending each respective kernel matrix structure to a distinct cell along a first dimension of the systolic array; for each depth level, sending the respective matrix of distinct activation inputs to a distinct cell along a second dimension of the systolic array; causing the systolic array to generate an accumulated output from the respective matrices sent to the cells; and generating the layer output from the accumulated output.