Edge Computing System for Low Latency Deep Learning Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Smart devices, such as home security cameras, face latency and privacy concerns due to the need for remote processing of image and video data, which can lead to impractical results and security risks, especially when transmitting sensitive data over networks.
Innovation Solution
An edge computing system that performs computations locally, using specialized hardware like tensor processing units and GPUs, to reduce latency and enhance security by controlling access to sensitive data, ensuring that only result data is returned to applications without exposing the raw video or image data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If remote processing is used for image and video data, then processing capability is improved, but latency increases and security risks worsen
Solution Approach 1:
The system segments processing tasks by separating computationally intensive deep learning inference (performed locally on the edge device) from data transmission and cloud processing. This allows the device to perform basic computer vision tasks autonomously without requiring continuous cloud connectivity, thereby reducing latency while maintaining processing capability for critical local operations.
Solution Approach 2:
The patent implements local quality by enabling edge devices to perform specific processing operations locally using onboard Tensor processing units and GPUs. The system provides different processing quality levels: basic processing locally for low-latency requirements and advanced processing via cloud for high-compute needs, allowing each device to operate at the appropriate quality level for its specific requirements.
2Power
If remote processing is used for image and video data, then processing capability is improved, but security risks worsen
Solution Approach 1:
The system introduces an intermediary security layer through the edge computing platform that acts as a mediator between local devices and cloud services. This intermediary controls what data is transmitted to the cloud, ensuring that only processed and anonymized results are sent, while raw sensitive data remains local. The intermediary also provides authentication and authorization mechanisms that control access to processing resources.
Solution Approach 2:
The patent extracts sensitive data handling from the cloud processing pipeline by implementing data extraction at the edge device level. The system extracts and processes only the necessary information locally (such as detecting objects or recognizing faces) before any data transmission occurs, thereby removing sensitive raw data from the transmission path and reducing security risks associated with cloud storage and processing.
3Power
If raw video or image data is transmitted to cloud services, then processing capability is improved, but privacy worsens
Solution Approach 1:
The system performs preliminary action by conducting deep learning inference and data processing locally on the edge device before any data is transmitted to cloud services. This preliminary processing extracts meaningful information (such as object detection results or facial recognition) from the raw data, allowing the system to send only the processed results to the cloud rather than the raw video or image data, thereby preserving privacy while maintaining processing capability.
4Productivity
If specialized hardware like Tensor processing units is used, then processing efficiency is improved, but device complexity worsens
Solution Approach 1:
The patent applies universality by designing the edge computing platform to support multiple processing workloads through a unified architecture. The same hardware platform with Tensor processing units can handle various tasks including computer vision, natural language processing, and machine learning inference. This multi-functionality allows the system to achieve high processing efficiency for specialized tasks while avoiding the need for separate dedicated hardware for each function, thereby managing device complexity.
Data Source
AI summary
Devices and techniques are generally described for an edge computing system. In various examples, a first application executed by a first virtual machine may generate a configuration file specifying a first machine learning task and first data. The first virtual machine may send the configuration file to a second virtual machine. The second virtual machine may generate first data effective to cause the first machine learning task to be executed by a third virtual machine. In various examples, the second virtual machine may retrieve the first data. In at least some examples, the second virtual machine may send the first data and the second data to a third virtual machine configured with access to a graphical processing unit.