Elementwise Neural Network Accelerator With Specialized ALU Pipelines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network accelerators are limited by the inability to efficiently perform a variety of elementwise operations in different formats, leading to inefficiencies and delays when executing neural networks that require operations not supported by their elementwise operations hardware accelerators.
Innovation Solution
The introduction of an elementwise operations hardware accelerator with multiple arithmetic logic unit (ALU) blocks capable of performing diverse elementwise operations in various formats, including bitwise, format conversion, and floating-point operations, along with a control module to dynamically manage these operations, allowing for more flexible and efficient neural network execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a single ALU block is used in existing neural network accelerators, then the device complexity is reduced, but the adaptability to perform various elementwise operations in different formats deteriorates
Solution Approach 1:
The patent implements multiple ALU blocks (first ALU block for first format, second ALU block for second format) where each block is specialized for specific operations in its designated format. This multi-functionality approach allows the accelerator to handle diverse elementwise operations including bitwise, format conversion, and floating-point operations without requiring a single complex universal ALU, thus improving adaptability while controlling complexity through specialization.
2Productivity
If existing accelerators support only limited operation formats, then the device complexity is reduced, but the productivity when executing diverse neural networks deteriorates
Solution Approach 1:
The patent segments the computational functionality into multiple dedicated ALU blocks, each handling specific operation formats (first format, second format). This segmentation allows parallel processing of different operation types simultaneously, improving productivity when executing neural networks with diverse elementwise operations. The control module coordinates these segmented units to process different elements of input tensors in parallel, reducing overall execution time.
3Loss of time
If external assistance is used for unsupported operations, then the adaptability is maintained, but the loss of time and bandwidth overhead increase
Solution Approach 1:
The patent implements self-service capability within the accelerator by providing multiple ALU blocks that can directly handle various elementwise operations including format conversion, bitwise operations, and floating-point operations. This eliminates the need to offload unsupported operations to external processors, thereby reducing time delays and bandwidth overhead. The control module manages the internal ALU blocks to process operations that would otherwise require external assistance.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An elementwise operations hardware accelerator for use in a neural network accelerator. The elementwise operations hardware accelerator comprises one or more processing pipelines and a control module. Each processing pipeline includes: an arithmetic logic unit module comprising a plurality of different arithmetic logic unit blocks, each arithmetic logic unit block of the plurality of arithmetic logic unit blocks configured to receive one or more inputs, selectively perform one or more elementwise operations on the one or more inputs, and output a result of the one or more elementwise operations; and an interconnection module configured to receive elements of one or more input tensors and selectively provide the elements of at least one of the one or more input tensors to an arithmetic logic unit block of the plurality of arithmetic logic unit blocks as an input; The control module is configured to receive a set of commands identifying an arithmetic logic unit block of the plurality of arithmetic logic unit blocks and one or more elementwise operations to be performed by the identified arithmetic logic unit block and control the operation of the one or more processing pipelines to cause the identified arithmetic logic unit block to perform the identified one or more elementwise operations.