Computation Engine Extract Instructions Minimize Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern computing systems, especially battery-powered devices, face performance issues and high power consumption due to the low efficiency of general-purpose processors in handling workloads like LSTM algorithms, which require numerous computations and frequent memory accesses, leading to significant latency and power consumption.
Innovation Solution
A computation engine is introduced that offloads computations from the processor, performing operations on input vectors within its memory and accumulating results, with extract instructions to minimize memory access, allowing additional processing without moving results to main memory, thus reducing main memory latency and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If general purpose processors are used to perform computations on large data sets, then the computations can be performed, but memory access latency increases and power consumption increases
Solution Approach 1:
The system segments computation and memory access operations by introducing a dedicated computation engine that can perform computations on data already loaded into its local input memories, separating the computation workload from the processor and reducing the frequency of main memory accesses
Solution Approach 2:
The computation engine acts as an intermediary between the processor and main memory, loading data from main memory into its local input memories once, then performing multiple computations on this data without requiring repeated main memory accesses, thus reducing memory latency
2Productivity
If general purpose processors are used to perform computations on large data sets, then the computations can be performed, but power consumption increases
Solution Approach 1:
The system segments the computation workload from the power-consuming main memory access operations by using a dedicated computation engine with local memory, so that computations can be performed without repeatedly accessing main memory, thereby reducing overall power consumption
Solution Approach 2:
The computation engine serves as an intermediary that buffers data in its local input memories, eliminating the need for repeated high-power main memory accesses during computation iterations, thus reducing power consumption while maintaining computational throughput
3Reliability
If results are moved to main memory after each computation, then computation results can be stored, but memory access latency increases
Solution Approach 1:
The system extracts the result storage function from main memory by providing dedicated output memory within the computation engine, allowing intermediate results to be stored locally without accessing main memory, thus reducing memory access latency while maintaining result integrity
Solution Approach 2:
The computation engine's output memory acts as an intermediary storage between computations and main memory, allowing results to be accumulated and stored locally before being transferred to main memory only when necessary, thereby reducing frequent memory access latency
Data Source
AI summary
In an embodiment, a computation engine may offload work from a processor (e.g. a CPU) and efficiently perform computations such as those used in LSTM and other workloads at high performance. In an embodiment, the computation engine may perform computations on input vectors from input memories in the computation engine, and may accumulate results in an output memory within the computation engine. The input memories may be loaded with initial vector data from memory, incurring the memory latency that may be associated with reading the operands. Compute instructions may be performed on the operands, generating results in an output memory. One or more extract instructions may be supported to move data from the output memory to the input memory, permitting additional computation on the data in the output memory without moving the results to main memory.