Collaborative Neural Network Adaptation for Edge Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks face challenges in achieving temporal consistency and efficient online learning on resource-constrained devices due to heavy computation requirements, particularly in video-based semantic segmentation, where methods often lack consistency between frames and incur excessive computational costs.
Innovation Solution
The implementation of a collaborative inference scheme that splits the computational load between a device and a cloud, using a generic neural network (GNN) and an efficient neural network (ENN), where ENN is deployed on the device and GNN on the cloud, with online learning activated based on performance criteria such as temporal consistency, and weight updates are communicated to adapt the ENN.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If online learning is activated to improve temporal consistency, then temporal consistency is improved, but computational overhead increases
Solution Approach 1:
The neural network is divided into two separate networks: a generic neural network (GNN) deployed on the cloud and an efficient neural network (ENN) deployed on the device. The GNN performs heavy computational tasks for online learning and weight updates, while the ENN handles inference on the device. This segmentation resolves the contradiction by moving computationally intensive operations to the cloud while maintaining temporal consistency through collaborative inference.
Solution Approach 2:
A collaborative inference framework acts as an intermediary between the device and cloud. The framework enables the ENN on the device to work together with the GNN on the cloud, allowing weight updates from the GNN to be communicated to the ENN. This intermediary mechanism enables online learning to improve temporal consistency without requiring the device to perform all computations locally, thus reducing computational overhead.
2Stability of the object's composition
If a generic neural network is deployed on the cloud for collaborative inference, then temporal consistency is improved, but network complexity increases
Solution Approach 1:
The system segments the neural network into two distinct components: a GNN for cloud-based collaborative inference and an ENN for device-based inference. This segmentation allows each network to be optimized for its specific function, with the GNN handling complex computations for temporal consistency while the ENN maintains simplicity for edge deployment.
Solution Approach 2:
The ENN on the device is essentially a copy or compressed version of the GNN on the cloud. The ENN receives weight updates from the GNN through collaborative inference, allowing it to replicate the GNN's performance for temporal consistency without requiring the full complexity of the GNN to be deployed on the device.
3Use of energy by moving object
If efficient neural network is deployed on the device, then computational overhead is reduced, but performance may deteriorate
Solution Approach 1:
The collaborative inference framework establishes a feedback loop where the GNN on the cloud receives inference results from the ENN on the device and communicates updated weights back to the ENN. This feedback mechanism ensures that the ENN continuously improves its performance by incorporating learnings from the GNN, maintaining high performance while operating with reduced computational overhead on the device.
Solution Approach 2:
The collaborative inference framework serves as an intermediary that bridges the ENN on the device and the GNN on the cloud. It enables the ENN to access the GNN's computational resources and updated weights through communication interfaces, ensuring that the ENN maintains high performance standards while operating efficiently on the device with limited computational resources.
Data Source
AI summary
An apparatus may be configured to: process at least one input with an efficient neural network; determine at least one performance criteria for the efficient neural network; and activate online learning for the efficient neural network based, at least partially, on the at least one performance criteria. An apparatus may be configured to: receive, from an efficient neural network, at least one video frame or at least one feature; determine at least one inference result based, at least partially, on the at least one video frame or the at least one feature; and transmit, to the efficient neural network, the at least one inference result.