Input Selector Circuit for Neural Network Processor Row Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network processors face low utilization of their computational arrays due to inefficient use of rows in the first layer of neural networks, leading to suboptimal performance in tasks like image recognition and machine translation, where only a few input feature maps occupy the initial rows, resulting in idle resources and reduced overall processing efficiency.
Innovation Solution
The implementation of an input selector circuit that allows for data duplication across multiple rows of the processing engine array, enabling different filter elements to be applied in parallel, thereby increasing the utilization of the computational array by replicating input feature maps and applying filters simultaneously across multiple rows, without the need for additional circuitry or significant memory bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is not duplicated across multiple rows, then memory bandwidth is conserved, but computational array utilization is low
Solution Approach 1:
The input selector circuit creates copies of input data from a single memory location and distributes them to multiple rows of the computational array simultaneously. This allows the same input feature map to be processed by multiple filter elements in parallel across different rows, increasing computational array utilization without requiring proportional increases in memory bandwidth, as the copying is performed by the selector circuit rather than through additional memory accesses
2Productivity
If more rows are actively engaged in processing, then processing efficiency is improved, but circuit complexity increases
Solution Approach 1:
The input selector circuit serves multiple functions: it selects input data for each row, duplicates data across rows, and routes data to appropriate destinations. This multi-functional approach allows a single circuit component to enable parallel processing across multiple rows without requiring separate control logic for each row, thereby improving processing efficiency while limiting the increase in circuit complexity
Solution Approach 2:
The input selector circuit acts as an intermediary between the memory subsystem and the computational array rows. It receives input data from memory and intelligently distributes it to multiple rows based on processing requirements, enabling coordinated parallel processing without direct complex interconnections between all components, thus improving efficiency while managing circuit complexity
Data Source
AI summary
Provided are systems and methods for operating a neural network processor, wherein the processor includes an input selector circuit that can be configured to select the data that will be input into the processor's computational array. In various implementations, the selector circuit can determine, for a row of the array, whether the row input will be the output from a buffer memory or data that the input selector circuit has selected for a different row. The row can receive an input feature map from a set of input data or an input feature map that was selected for inputting into a different row, such that the input feature map is input into more than one row at a time. The selector circuit can also include a delay circuit, so that the duplicated input feature map can be input into the computational array later than the original input feature map.