Edge-Cloud Inference Network for Latency and Robustness Trade-offs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In computing networks, there is a balance to be struck between inference robustness, processing power, and latency, where edge-based machine learning models may provide low latency but reduced robustness due to limited computing power, while cloud-based models offer greater power but increased latency.
Innovation Solution
A computing network architecture that includes a sensor, a first processor near the sensor for real-time inference and a second processor further away for additional, more robust inference processing, triggered when necessary, allowing for the aggregation of inferences from both models to enhance decision-making.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If machine learning models are deployed at the edge of computing networks to process sensor data in real time, then latency is reduced and real-time inference is achieved, but inference robustness deteriorates due to limited computing power at edge locations
Solution Approach 1:
The system segments the inference process into two parts: a first machine learning model deployed at the edge location (first computing network location) for real-time initial inference, and a second machine learning model deployed at a remote location (second computing network location) for more robust secondary inference. This segmentation allows each model to operate in its optimal environment while collectively solving the contradiction between latency and robustness.
Solution Approach 2:
The system adds a spatial dimension to the inference architecture by introducing a second computing network location remote from the sensor. Instead of relying on a single edge location, the system distributes inference capabilities across multiple locations at different distances from the sensor, enabling a trade-off between proximity (low latency) and computing power (high robustness).
2Reliability
If multiple machine learning models are used to improve inference robustness through ensemble voting, then inference quality improves, but processing power requirements and system complexity increase
Solution Approach 1:
The ensemble of multiple machine learning models is segmented across two different computing network locations. The first model at the edge location handles real-time processing, while the second model at the remote location provides additional robustness. This spatial segmentation of the ensemble reduces the complexity burden at any single location while maintaining the benefits of multiple models.
Solution Approach 2:
The system implements a triggered mechanism where the second machine learning model at the remote location is not always active but is triggered based on the output of the first model. This partial action approach maintains inference robustness through ensemble voting only when necessary, reducing the overall processing power requirements and system complexity compared to running all models continuously.
Data Source
AI summary
A computing network has a sensor, a first processor in a first computing network location, and a second processor in a second computing network location, the second computing network location further from the sensor than the first computing network location. The first processor is configured to receive sensor data from the sensor and configured to operate a first machine learning model to make a first inference based on the sensor data. The second processor is configured to receive the sensor data and to operate a second machine learning model to make a second inference based on the sensor data in response to a trigger. The computing network is configured to collate and process the first and second inferences to make an aggregated inference.