Edge-Server Inference With Reliability-Based Feature Offloading
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The edge/cloud network faces challenges in real-time performance due to high communication costs and delays, particularly in applications like automatic analysis of surveillance camera images and automatic driving, where the edge device's computation resources are insufficient for large computation loads.
Innovation Solution
A processing system that utilizes an edge device with a lightweight model (DNN1) and a server device with a high-accuracy model (DNN2), where the edge device performs inference using DNN1 and transfers feature maps to the server device for further processing by DNN2 when reliability thresholds are not met, reducing data transfer and delay.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If processing with large computation amount is performed by the edge device, then accuracy can be improved, but processing time increases and real-time performance deteriorates
Solution Approach 1:
The inference processing is segmented into two parts: feature extraction performed at the edge device using a lightweight model, and classification performed at the cloud server using a high-accuracy model. This segmentation allows the edge device to handle time-sensitive preprocessing locally while offloading computationally intensive classification to the cloud, thereby maintaining real-time performance while achieving high accuracy.
Solution Approach 2:
The edge device performs only partial inference (feature extraction) rather than complete inference, using a lightweight model that provides sufficient but not excessive processing capability. This partial action enables the system to meet real-time requirements while still achieving acceptable accuracy for time-critical decisions, with the remaining classification accuracy provided by the cloud server.
2Speed
If all processing is performed by the edge device, then real-time performance can be improved, but accuracy deteriorates due to limited computation resources
Solution Approach 1:
The inference processing is segmented into two parts: feature extraction performed at the edge device using a lightweight model, and classification performed at the cloud server using a high-accuracy model. This segmentation allows the edge device to handle time-sensitive preprocessing locally while offloading computationally intensive classification to the cloud, thereby maintaining real-time performance while achieving high accuracy.
3Measurement precision
If processing is offloaded to the cloud server, then accuracy can be improved, but communication cost and delay increase
Solution Approach 1:
The system extracts and transmits only the essential feature representation from the edge device to the cloud server, rather than transmitting raw sensor data or intermediate processing results. This extraction of only necessary information minimizes communication overhead and data transfer costs while still enabling the cloud server to perform accurate classification.
Solution Approach 2:
The edge device performs preliminary feature extraction and processing before transmitting data to the cloud server. This preliminary action reduces the amount of data that needs to be communicated and prepares the data in a form that is optimized for cloud processing, thereby reducing communication cost and delay while maintaining inference accuracy.
4Speed
If a lightweight model is used at the edge device, then real-time performance can be improved, but accuracy deteriorates
Solution Approach 1:
The inference processing is segmented into two parts: feature extraction performed at the edge device using a lightweight model, and classification performed at the cloud server using a high-accuracy model. This segmentation allows the edge device to handle time-sensitive preprocessing locally while offloading computationally intensive classification to the cloud, thereby maintaining real-time performance while achieving high accuracy.
Data Source
AI summary
A processing system is performed by using an edge device and a server device, wherein the edge device includes first processing circuitry configured to extract a feature amount of processing target data by using a first model and execute inference processing on the processing target data on a basis of the extracted feature amount, and output an inference result in a case where reliability of the inference result exceeds a threshold, and output the feature amount of the processing target data to the server device in a case where the reliability is equal to or less than the threshold, and the server device includes second processing circuitry configured to execute inference processing on the processing target data on the basis of the feature amount of the processing target data output from the edge device by using a second model having higher inference accuracy than the first model.