Computation Engine Vector Interleaving for LSTM Power
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
General-purpose processors, even with vector instructions, exhibit low performance and high power consumption on workloads like LSTM algorithms, which are problematic for computing systems, especially battery-powered devices due to their low performance and high power consumption.
Innovation Solution
A computation engine offloads processor work and efficiently performs computations by converting and interleaving vector elements to specific precisions as specified by instructions, supporting high precision computations and maintaining relevant precisions outside the engine, thereby optimizing performance and reducing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If general-purpose processors perform computations on small numbers with low precision, then power consumption is high, but performance is low
Solution Approach 1:
The system segments computation tasks by separating high-precision operations (performed outside the computation engine) from low-precision operations (performed inside the computation engine). This allows the computation engine to handle throughput-bound workloads efficiently using lower precision, while accuracy-bound workloads are offloaded to external high-precision units, resolving the contradiction between performance and power consumption.
Solution Approach 2:
The system dynamically changes the precision parameter of computations based on workload characteristics. The computation engine uses lower precision (e.g., 8-bit integers) for throughput-bound workloads to reduce power consumption, while maintaining the ability to switch to higher precision for accuracy-bound workloads, thus optimizing the power-performance tradeoff.
2Measurement precision
If vector elements are converted to higher precision, then computation accuracy is improved, but memory bandwidth and processing complexity increase
Solution Approach 1:
The system applies different precision levels to different parts of the computation based on local requirements. Throughput-bound computations use lower precision locally within the computation engine, while accuracy-bound computations are handled by external high-precision units. This local differentiation of precision quality improves accuracy where needed without unnecessarily increasing overall processing complexity.
Solution Approach 2:
The computation engine acts as an intermediary between low-precision high-throughput operations and high-precision accuracy-critical operations. It receives low-precision inputs, performs intermediate computations, and selectively offloads operations requiring high precision to external units, thereby managing the tradeoff between accuracy and complexity through this intermediary layer.
3Measurement precision
If all computations are performed with high precision, then accuracy is maintained, but power consumption and processing time increase
Solution Approach 1:
The system applies partial precision to computations based on actual needs. Rather than uniformly applying high precision to all operations, it uses lower precision for throughput-bound computations where full accuracy is not critical, and reserves high precision for accuracy-bound operations. This partial application of high precision reduces processing time while maintaining accuracy where necessary.
Data Source
AI summary
In an embodiment, a computation engine may perform computations on input vectors having vector elements of a first precision and data type. The computation engine may convert the vector elements from the first precision to a second precision and may also interleave the vector elements as specified by an instruction issued by the processor to the computation engine. The interleave may be based on a ratio of a result precision and the second precision. An extract instruction may be supported to extract results from the computations and convert and deinterleave the vector elements to provide a compact result in a desired order.