Aperture Function IC for Neural Network Row Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer systems for matrix operations in neural networks are inefficient due to sequential multiplication of weights with input values, leading to high time and cost requirements, and lack optimization in the order of mathematical processes.
Innovation Solution
A system utilizing integrated circuits (ICs) that apply an aperture function to input data streams ordered row by row, retaining partial values for each column and providing them back to complete output values, allowing for simultaneous multiplication of weights with input values across multiple rows, reducing the need for redundant RAM access and increasing processing speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If sequential multiplication of weights with input values is used, then implementation simplicity is maintained, but processing time and cost increase significantly
Solution Approach 1:
The patent segments the input data into multiple rows and processes them simultaneously through parallel multiplication operations. Each row of input values is multiplied by the weight values in parallel, dividing the single sequential task into multiple concurrent sub-tasks that can be executed simultaneously, thereby increasing processing throughput.
Solution Approach 2:
The patent transitions from one-dimensional sequential processing to two-dimensional parallel processing by utilizing the row dimension of matrix data. Multiple rows are processed simultaneously across the horizontal dimension while maintaining the vertical dimension of weight multiplication, effectively adding a spatial dimension to the computation to achieve parallelism.
2Productivity
If extensive RAM buffering is used to support parallel processing, then throughput increases, but power consumption and cost increase
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing partial row products in compact buffers before final summation. Rows are partially processed and their results cached in small, efficient memory structures, allowing subsequent rows to be processed without requiring large buffering capacity, thus enabling parallel processing with minimal RAM usage.
Solution Approach 2:
The patent maintains continuity of useful action by processing rows in a continuous pipeline where partial results are immediately available for the next computation stage. This continuous flow eliminates the need for large buffers to hold idle data, as each piece of data is actively processed through multiple stages without interruption or idle storage, reducing memory requirements while maintaining high throughput.
3Loss of time
If redundant RAM access is required for sequential processing, then memory access simplicity is maintained, but processing time increases
Solution Approach 1:
The patent merges multiple RAM access operations into a single unified access pattern. Instead of sequentially accessing memory for each row multiplication, the system combines all row data into a parallel processing pipeline that performs multiplications simultaneously, consolidating multiple memory access sequences into one coordinated operation that reduces total access time despite increased data flow complexity.
Solution Approach 2:
The patent addresses redundant memory access by transitioning to parallel processing across the row dimension. Multiple rows are loaded and processed simultaneously, transforming the time-consuming sequential access pattern into a spatially distributed parallel access pattern that reduces the temporal dimension of memory access while introducing parallel data flow paths.
Data Source
AI summary
A system processing a stream of input data ordered row by row from a data array has a first integrated circuit (IC) adapted to apply an aperture function to the stream of input data, to produce an output data stream, and a second IC coupled to the first IC, the second IC adapted to manage context from row to row, retaining partial values as computed by the aperture function for each column along a row, and providing the partial values back to the aperture function for subsequent rows as needed to complete output values.