Consensus-Based Neural Network Sizing for IoT Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural networks require significant processing power, memory, and bandwidth, making them impractical for applications like IoT devices that need to operate at low voltage and high efficiency without sacrificing accuracy, especially when dealing with high-resolution images or videos.
Innovation Solution
The approach involves using a linear chain of increasingly complex neural networks trained on progressively larger inputs to reach a consensus point, where smaller neural networks can achieve acceptable inference outputs with better performance and power efficiency, thereby optimizing resource utilization and reducing over-engineering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If larger and more complex neural networks are used, then inference accuracy is improved, but processing power requirements and resource consumption increase
Solution Approach 1:
The patent segments the neural network evaluation process into multiple networks of varying sizes and complexities. Instead of using a single large network, it employs a sequence of networks (e.g., small, medium, large) that process inputs progressively. This segmentation allows the system to achieve high accuracy through consensus among multiple networks while avoiding the need for any single network to be excessively large and power-intensive.
Solution Approach 2:
The patent applies partial action by using only the necessary amount of network complexity required to achieve consensus. Rather than always deploying the largest possible network, the system uses smaller networks when sufficient and escalates to larger networks only when needed to resolve uncertainty. This partial approach reduces average power consumption while maintaining accuracy when required.
2Measurement precision
If larger and more complex neural networks are used, then inference accuracy is improved, but memory constraints are exceeded
Solution Approach 1:
The patent divides the memory burden by distributing neural network models across different size categories. Instead of loading one massive network that consumes excessive memory, the system loads multiple smaller networks sequentially or in parallel, each requiring less memory individually. This segmentation enables the system to operate within memory constraints while still achieving high accuracy through the collective output of multiple networks.
3Measurement precision
If larger and more complex neural networks are used, then inference accuracy is improved, but time to train increases
Solution Approach 1:
The patent applies preliminary action by pre-training multiple neural networks of different sizes before deployment. These networks are trained in advance on the training dataset, and their weights are frozen during inference. This preliminary training allows the system to quickly perform inference using pre-processed models without requiring real-time training, thus reducing the time penalty associated with having multiple networks while maintaining the ability to achieve high accuracy through their combined output.
4Productivity
If smaller neural networks are used, then processing efficiency is improved, but inference accuracy decreases
Solution Approach 1:
The patent merges the outputs of multiple small-to-medium neural networks to achieve the accuracy that would otherwise require a single large network. By combining the inference results from several smaller networks through consensus mechanisms (e.g., voting, averaging probabilities), the system achieves high accuracy while maintaining the processing efficiency and low resource consumption of smaller individual networks.
Solution Approach 2:
The patent implements feedback by using the outputs of smaller networks to determine whether larger networks are needed. If smaller networks reach consensus on an inference result, the system accepts that result without further processing. If they disagree or show uncertainty, the system activates larger networks to resolve the discrepancy. This feedback mechanism ensures that smaller networks handle most cases efficiently while larger networks intervene only when necessary to maintain accuracy.
Data Source
AI summary
A system includes a determination component that determines output for successively larger neural networks of a set; and a consensus component that determines consensus between a first neural network and a second neural network of the set. A linear chain of increasingly complex neural networks trained on progressively larger inputs is utilized (e.g., increasingly complex neural networks is generally representative of increased accuracy). Outputs of progressively networks are computed until a consensus point is reached—where two or more successive large networks yield a same inference output. At such point of consensus the larger neural network of the set reaching consensus can be deemed appropriately sized (or of sufficient complexity) for a classification task at hand.