Variable-Accuracy Computing for Edge LLM Power-Performance Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing computational demands and energy requirements of artificial neural networks, particularly Large Language Models (LLMs), necessitate a solution for efficient and flexible operation in edge devices, balancing performance and power consumption.
Innovation Solution
A computing system with a controller that adjusts the accuracy of operations in a computation unit based on a tunable performance parameter, allowing control over power consumption and performance metrics such as perplexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If computation accuracy is increased to improve LLM performance, then performance metric (e.g., perplexity) is improved, but power consumption increases
Solution Approach 1:
The system dynamically adjusts the computation accuracy level based on runtime conditions and performance requirements. The controller monitors the performance metric and power consumption, then adaptively selects appropriate accuracy levels for different computation units or different time periods, allowing the system to optimize the trade-off between performance and power consumption in real-time
Solution Approach 2:
Different computation units within the neural network are assigned different accuracy levels based on their specific function and importance. Critical computation units maintain high accuracy to ensure overall performance, while less critical units operate at lower accuracy levels to reduce power consumption. This selective approach allows the system to achieve acceptable overall performance with reduced total power consumption
2Use of energy by moving object
If computation accuracy is decreased to reduce power consumption, then power consumption is reduced, but performance metric deteriorates
Solution Approach 1:
The system changes the accuracy parameter of computation units based on controlled conditions. The controller adjusts parameters such as precision, bit-width, or computational complexity of different computation units according to the desired performance level and power constraints, enabling flexible operation across different accuracy levels without hardware modification
Solution Approach 2:
The system applies reduced accuracy computation selectively to certain computation units or certain operations within the neural network, rather than uniformly reducing accuracy across all units. This partial application allows the system to achieve power savings while maintaining sufficient performance through the contributions of critical high-accuracy units
3Reliability
If fixed high accuracy is used to ensure performance, then performance metric is maintained, but flexibility is reduced
Solution Approach 1:
The system transitions from fixed accuracy configuration to dynamic accuracy adjustment. The controller enables real-time switching between different accuracy levels based on runtime conditions, user requirements, or performance monitoring, allowing the same hardware to adapt to various workloads and power constraints without redesign
4Reliability
If centralized cloud computation is used to achieve high performance, then performance metric is improved, but device autonomy and speed are reduced
Solution Approach 1:
The edge device is equipped with local neural network computation units that can autonomously perform inference tasks without requiring constant cloud connectivity. The device self-serves its computation needs by executing neural networks locally, achieving both improved inference speed and maintained performance through on-device computation capabilities
Data Source
AI summary
A computing system comprising: a computation unit configured to receive a series of input data values and generate a series of output data values by performing operations on at least one received input data value and/or generated output data value; an input to receive a tunable performance parameter separate to the series of input data values; and a controller, wherein the controller is configured, as a function of the received tunable performance parameter, to issue a control signal to the computation unit to control a level of accuracy of the operations and thereby affect a performance metric of the computation unit.