Mass Multiplier Circuit for Neural Network Pipelining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer processors, including CPUs, GPUs, and TPUs, do not efficiently utilize the fact that in neural network operations, one weight factor is common for all weights applied to an input channel during processing, leading to suboptimal performance in matrix operations such as convolutional neural networks, where numerous multiplications are required.
Innovation Solution
Implementing a mass multiplier circuit that performs all multiplications by a set of common weights simultaneously for each input channel, reducing the need for sequential operations and minimizing RAM access, and using an everted order of operations to process inputs only once and retain partial sums in hardware registers, allowing for higher throughput and reduced power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If conventional sequential multiplication operations are used in neural networks, then processing can be performed with standard CPU/GPU/TPU architectures, but processing speed is limited and RAM access contention occurs
Solution Approach 1:
The patent segments the weight matrix into multiple sub-matrices and processes different segments in parallel through multiple processing elements. This allows simultaneous multiplication operations on different portions of the input data, dramatically increasing processing speed while distributing computational complexity across multiple simple units rather than requiring a single complex processor
Solution Approach 2:
The patent transforms the traditional sequential multiplication process into a parallel operation by adding a spatial dimension through multiple processing elements arranged in a pipeline. Data flows through multiple stages simultaneously, converting a time-sequential operation into a space-parallel operation that achieves higher throughput
2Productivity
If multiple RAM accesses are performed for each input value in neural network operations, then all weight multiplications can be completed, but RAM contention increases and latency increases
Solution Approach 1:
The patent performs preliminary loading of weight values into on-chip registers before the multiplication process begins. This pre-fetching and caching of frequently accessed weight data eliminates repeated RAM accesses during the computation, reducing latency and allowing continuous high-speed processing without RAM contention bottlenecks
Solution Approach 2:
The patent introduces on-chip registers as intermediary storage between RAM and the processing elements. These registers act as a buffer that holds weight values during computation, eliminating the need for repeated RAM accesses and reducing the latency caused by memory contention while maintaining high throughput
3Productivity
If sequential processing of input values is used, then power consumption is reduced, but processing throughput is limited
Solution Approach 1:
The patent divides the computational workload into segments processed by multiple dedicated processing elements operating in parallel. Each element performs simple multiplication and accumulation operations simultaneously on different data segments, achieving high throughput while keeping individual element power consumption low due to the simplicity of each unit's operation
Data Source
AI summary
An integrated circuit (IC) implements an M by N aperture function over an R by C source array. The IC has an input port receiving an ordered stream of independent input values, an output port producing an output stream, a mass multiplier circuit multiplying inputs by weights, producing streams of products on pathways on the IC, an M by N array of compositor circuits on the IC, single dedicated pathways between compositors, delay circuits, a finalization circuit, and a control circuit operating counters and producing control signals. The compositors combine the values received from product pathways, further combine that result to an initial value or to a value from an adjacent compositor upstream, or to a value from a delay circuit. Upon a last downstream compositor producing a complete composition of values, that value is passed to the finalization circuit, which posts a result to the output port.