Low Energy Programmable Vector Processing Unit for Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI processing methods, particularly in neural networks, are inefficient in terms of power consumption and speed due to the complexity of backend operations such as pooling, ReLu, Tanh, Sigmoid, and element-wise operations, which require numerous serial operations and are not power efficient.
Innovation Solution
A low-energy programmable vector processing unit is introduced, utilizing a Single Instruction Multiple Data (SIMD) datapath that performs various backend processing functions with a single instruction, allowing parallel execution of operations like pooling, permutation, sigmoid/tanh, and element-wise operations across multiple processing cells.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If current approaches are used for pooling, ReLu, Tanh, Sigmoid, and element wise operations, then these operations can be performed using common hardware algorithms, but the processing is complex, power inefficient, and slow requiring numerous reads and writes and many serial operations
Solution Approach 1:
The processing unit is divided into multiple processing cells (e.g., 32 cells) that can operate in parallel. Each cell handles a portion of the data independently, allowing simultaneous execution of the same operation on multiple data elements. This segmentation transforms serial processing into parallel processing, dramatically improving speed while maintaining algorithm simplicity.
Solution Approach 2:
The invention introduces a new dimension of parallelism by organizing processing elements spatially in an array structure. Instead of sequentially executing operations in time, the system distributes operations across multiple spatial processing cells, each capable of independent computation. This dimensional transformation from temporal to spatial processing resolves the contradiction between speed and complexity.
2Use of energy by moving object
If current approaches are used for pooling, ReLu, Tanh, Sigmoid, and element wise operations, then these operations can be performed using common hardware algorithms, but the processing requires numerous reads and writes and many serial operations
Solution Approach 1:
Multiple operations (pooling, ReLu, Tanh, Sigmoid, element-wise operations) are merged into a single unified processing structure. The processing cells can perform different operations simultaneously on different data streams, eliminating the need for separate hardware blocks and reducing overall power consumption while maintaining high processing throughput.
Solution Approach 2:
The processing cells operate continuously without idle periods between operations. Data flows continuously through the processing pipeline, with each cell constantly performing computations. This eliminates gaps in useful action and maximizes both energy efficiency and processing speed by keeping all processing resources actively engaged.
3Productivity
If a SIMD datapath is used to perform backend processing functions with a single instruction, then processing speed and power efficiency are improved, but the device requires a programmable controller and multiple processing cells
Solution Approach 1:
The processing cells are designed as universal units capable of performing multiple different operations (pooling, permutation, sigmoid/tanh, element-wise operations). Each cell can be configured via control signals to execute any of these operations, eliminating the need for dedicated hardware for each function. This multi-functionality achieves high processing efficiency while controlling hardware complexity through standardization.
Data Source
AI summary
An approach is described for a method and apparatus for a low energy programmable vector processing unit for use in processing such as for example neural network backend processing. According to some embodiments, this approach provides a pooling/vector processing unit for performing backend processing that implements a single issue multiple data (SIMD) datapath that performs various backend processing functions using only a single instruction. For instance, the present approach provides an apparatus and method for execution of operations in parallel using a single issued instruction to a plurality of processing cells. In some embodiments, there are multiple groups of processing cells for performing different operations—e.g. pooling, permute, sigmoid/tanh, and element wise operations. In some embodiments, the single instructions are provided by a programmable controller within the pooling/vector processing unit for issuing instructions which manages the retrieval and issuance of instructions within the pooling/vector processing unit.