Strided Dot Product Computation Engine for LSTM Power Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
General-purpose processors, even with vector instructions, exhibit low performance and high power consumption on workloads that require massive computations, such as those used in LSTM algorithms, which is particularly problematic for battery-powered systems like mobile devices.
Innovation Solution
A computation engine offloads work from the processor and efficiently performs computations like dot products by using strided operations, where a subset of vector elements are selected for the dot product operation, reducing unnecessary computations and improving performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If general-purpose processors perform dot product computations using vector instructions, then the computations can be executed, but the performance is very low and power consumption is very high
Solution Approach 1:
The computation engine is segmented from the general-purpose processor, creating a dedicated hardware unit for specific computational tasks. This segmentation allows the computation engine to be optimized for specific operations like dot products and LSTM workloads, achieving high performance with lower power consumption compared to general-purpose processors
Solution Approach 2:
The computation engine acts as an intermediary between the processor and memory systems, handling computationally intensive tasks. It receives data from memory, performs optimized computations using dedicated hardware circuits, and returns results to the processor, thereby offloading work from the general-purpose processor and improving overall system efficiency
2Reliability
If all vector elements are processed in dot product operations, then complete computation is achieved, but unnecessary computations increase power consumption and reduce efficiency
Solution Approach 1:
The computation engine extracts and processes only the necessary subset of vector elements based on stride parameters. By taking out only the required elements for computation and skipping unnecessary ones, the system achieves energy efficiency while maintaining computation reliability for the needed data points
Solution Approach 2:
The system performs partial action by processing only the necessary portion of vector elements rather than all elements. The stride mechanism allows selective processing of elements at specific intervals, avoiding excessive computation on unnecessary data while ensuring all required computations are performed
Data Source
AI summary
In an embodiment, a computation engine may perform dot product computations on input vectors. The dot product operation may have a first operand and a second operand, and the dot product may be performed on a subset of the vector elements in the first operand and each of the vector elements in the second operand. The subset of vector elements may be separated in the first operand by a stride that skips one or more elements between each element to which the dot product operation is applied. More particularly, in an embodiment, the input operands of the dot product operation may be a first vector having second vectors as elements, and the stride may select a specified element of each second vector.