Artificial Neural Network Inference Control for Low-Power Edge Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
There is a challenge in balancing high performance and low power consumption in artificial neural network (ANN) systems for edge devices, particularly in always-on applications like mobile phones, which require efficient detection and processing of user commands while minimizing power usage.
Innovation Solution
An ANN computing system with a buffer, inference engine, and controller that adjusts operation modes based on an interest metric, transitioning to higher power modes when necessary to process input data effectively, using a subset of neural network layers and varying clock frequencies to optimize performance and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the inference engine operates in high-performance mode continuously, then processing capability is improved, but power consumption increases
Solution Approach 1:
The inference engine implements dynamic mode switching between low-power and high-performance operations based on real-time analysis of input data characteristics. The system transitions from a static operational state to a dynamic one where processing intensity adapts to the actual needs of the data being processed, thereby resolving the contradiction between continuous high performance and power consumption.
Solution Approach 2:
The system changes operational parameters (processing intensity, activation of neural network layers) based on the analysis of input data. By dynamically adjusting these parameters according to data characteristics, the system achieves high performance only when necessary while maintaining low power consumption during normal operation.
2Measurement precision
If all neural network layers are activated for processing, then accuracy is improved, but computational overhead increases
Solution Approach 1:
Different neural network layers are selectively activated based on the local characteristics of the input data. Rather than uniformly processing all data through the entire network, the system applies different processing intensities to different data streams, activating only the necessary layers for each specific input, thereby reducing overall computational overhead while maintaining accuracy where needed.
Solution Approach 2:
The system applies partial processing (using only necessary neural network layers) for most data inputs, and excessive processing (activating all layers) only when the data characteristics indicate high importance or ambiguity. This selective approach reduces average computational overhead while preserving detection accuracy for critical cases.
3Reliability
If the system processes all buffered data, then completeness is improved, but processing time increases
Solution Approach 1:
The system performs preliminary analysis of buffered data to identify characteristics that indicate potential user commands or significant events. By pre-processing and categorizing data before full inference processing, the system can quickly determine which buffered data requires complete processing, thereby reducing overall processing time while maintaining detection completeness for important inputs.
Solution Approach 2:
For data that preliminary analysis determines is unlikely to be significant (routine or irrelevant inputs), the system skips detailed processing and moves quickly through the buffer. This selective skipping reduces average processing time while ensuring that potentially important data receives complete processing attention.
Data Source
AI summary
The present disclosure relates to an artificial neural network (ANN) computing system comprising: a buffer configured to store data indicative of input data received from an input device; an inference engine operative to process data from the buffer to generate an interest metric for the input data; and a controller. The controller is operative to control a mode of operation of the inference engine according to the interest metric for the input data.