Neural Network Partitioning for Resource-Constrained Edge Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Devices with low computational resources, such as smart earbuds and IoT devices, face challenges in performing neural network training due to the high computational requirements, limiting their ability to improve performance over time.
Innovation Solution
The solution involves partitioning a neural network into an encoder and a predictor, where the predictor is trained locally on the device using captured data, while the encoder is trained remotely, optimizing the architecture based on available computational resources to enable efficient inference and training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If neural network training is performed on devices with low computational resources, then the device can learn from local data and improve performance, but the computational resource requirements exceed what is available on such devices
Solution Approach 1:
The neural network is segmented into two distinct parts: an encoder that extracts features from input data and a predictor that makes predictions based on those features. This segmentation allows the computationally intensive encoder to be trained remotely on powerful servers while only the lighter predictor needs to be trained locally on resource-constrained devices, resolving the contradiction between learning capability and computational resource requirements.
2Adaptability or versatility
If the entire neural network is trained locally, then the device can adapt to local data patterns, but the memory and computational requirements become too high for the device
Solution Approach 1:
The encoder component, which requires significant memory and computational resources for training, is extracted from the device and trained remotely on powerful servers. Only the essential predictor component remains on the device, dramatically reducing memory requirements while preserving the ability to adapt to local data patterns through continued training of the predictor using locally extracted features.
3Manufacturing precision
If data is transmitted for remote training, then the encoder can be trained with high computational resources, but transmission costs and data privacy concerns increase
Solution Approach 1:
The system implements local quality by performing feature extraction locally on the device using the encoder, then transmitting only the extracted features and predictions to the server for training. This approach maintains high training accuracy by utilizing the encoder's feature extraction capabilities while minimizing data transmission requirements and preserving data privacy, as raw sensor data never leaves the device.
4Ease of operation
If the neural network architecture is optimized for low-resource devices, then the device can perform inference, but the network performance and accuracy decrease
Solution Approach 1:
The system resolves the accuracy-capability contradiction by introducing a hierarchical dimension: the full-accuracy encoder is executed remotely on powerful servers for training and complex inference tasks, while a simplified predictor runs locally on resource-constrained devices for rapid inference. This dimensional separation allows each component to operate at its optimal performance level, with the local predictor achieving sufficient accuracy for real-time applications while the remote encoder provides high-accuracy processing when available.
Data Source
AI summary
A terminal apparatus comprising capturing data, transmitting information indicative of computational resources available at the apparatus for neural network training, receiving an encoder, defining one or more layers of artificial neurons, to be used as an input portion of a neural network receiving a predictor, defining one or more layers of artificial neurons, to be used as an output portion of the neural network; training the predictor, not the encoder, using at least some of the captured data; and performing inference on captured data using the neural network formed from the encoder and the predictor.