Runtime-Throttleable Neural Networks via Gating
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning algorithms, particularly artificial neural networks (ANNs), face challenges in training times, memory requirements, processor availability, battery power consumption, and energy efficiency, limiting their application in resource-constrained environments and dynamic conditions.
Innovation Solution
The development of runtime-throttleable neural networks that adaptively balance performance and resource use through a gating scheme, allowing for the selective deactivation of neural network components based on control signals, enabling efficient operation in varying conditions without affecting accuracy significantly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If neural network components are deactivated to reduce resource use, then energy consumption and computational resources are reduced, but processing speed and inference time increase
Solution Approach 1:
The neural network implements dynamic component deactivation where computational components are selectively activated or deactivated at runtime based on input characteristics and resource availability. The gating mechanism dynamically determines which components to activate, allowing the system to adapt processing speed and energy consumption in real-time rather than using a static configuration.
2Loss of energy
If neural network components are deactivated to conserve resources, then battery power consumption is reduced, but inference accuracy deteriorates
Solution Approach 1:
The gating mechanism applies local quality by selectively activating specific components of the neural network based on the characteristics of the input data and resource constraints. Different components are activated locally rather than uniformly across the entire network, allowing the system to maintain high accuracy for critical components while deactivating less important ones to conserve battery power.
3Device complexity
If a uniform neural network model is used for all components, then model complexity is reduced, but adaptability to different operating conditions decreases
Solution Approach 1:
The neural network is segmented into multiple independent components with separate trainable parameters, allowing each component to be selectively activated or deactivated. This segmentation enables the uniform model to adapt to different operating conditions by activating appropriate subsets of components without requiring multiple separate models, thus maintaining low model complexity while achieving high adaptability.
Data Source
AI summary
In general, the disclosure describes techniques for creating runtime-throttleable neural networks (TNNs) that can adaptively balance performance and resource use in response to a control signal. For example, runtime-TNNs may be trained to be throttled via a gating scheme in which a set of disjoint components of the neural network can be individually “turned off” at runtime without significantly affecting the accuracy of NN inferences. A separate gating neural network may be trained to determine which trained components of the NN to turn off to obtain operable performance for a given level of resource use of computational, power, or other resources by the neural network. This level can then be specified by the control signal at runtime to adapt the NN to operate at the specified level and in this way balance performance and resource use for different operating conditions.