Multi-task Segmented Learning Models for Edge Cloud Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models face significant computational overhead and latency issues due to the disparity in processing power and resources between cloud computing systems and edge devices, particularly in IoT and V2I services, where loading pre-trained weight sets and models incurs substantial overhead and latency, especially in real-time applications like security monitoring and traffic monitoring.
Innovation Solution
The solution involves segmenting a deep learning model into a central network and sub-networks, where sub-networks are executed on edge devices to perform initial computations and output intermediate data, which is then processed by the central network in the cloud, reducing the need for loading different weight sets and models, and using a shared weight set for heterogeneous tasks by training on feature embeddings that optimize for both similarity and distance between data points.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If pre-trained weight sets and models are loaded into memory from storage for each task, then the learning model can perform computations to receive input and output results, but the computational overhead and latency increase significantly
Solution Approach 1:
The learning model is divided into a shared backbone network and task-specific head networks. The backbone network is pre-trained once and shared across multiple tasks, while each task has its own lightweight head network. This segmentation allows the system to avoid loading complete pre-trained models for each task, reducing memory overhead and loading latency while maintaining task-specific performance.
Solution Approach 2:
The backbone network serves as a universal feature extractor that can be shared across multiple different tasks. Instead of training separate complete models for each task, the same backbone network processes features for all tasks, reducing redundant computations and memory requirements while maintaining the ability to perform multiple functions.
2Measurement precision
If specialized learning models and weight sets are deployed for each task, then the model can be optimized for particular features and algorithms, but the device complexity and resource requirements increase
Solution Approach 1:
The model architecture is segmented into a shared backbone and task-specific heads. The backbone contains the complex pre-trained weights for general feature extraction, while each task only requires a simple head network with fewer parameters. This reduces the complexity of deploying specialized models for each task while maintaining task-specific optimization through the head networks.
Solution Approach 2:
Multiple task-specific head networks are merged with a single shared backbone network. Instead of deploying separate complete models for each task, the system combines multiple heads that share the same backbone, reducing overall model complexity and resource requirements while maintaining the ability to perform multiple tasks with task-specific optimization.
3Power
If cloud computing systems with high processing power are used, then complex deep learning computations can be performed, but the latency between data collection at edge devices and output of results increases
Solution Approach 1:
The computation workload is segmented between edge devices and cloud systems. Lightweight head networks are deployed at edge devices to perform initial processing and filtering, while only the necessary features and results are transmitted to the cloud for processing by the shared backbone network. This segmentation reduces latency by performing computations closer to the data source while still utilizing cloud computing power for complex operations.
Solution Approach 2:
Edge devices perform preliminary processing using the head networks to extract and filter features before transmitting data to the cloud. This preliminary action reduces the amount of data that needs to be transmitted and processed centrally, improving overall system speed by preparing data in advance at the edge rather than waiting for cloud processing.
Data Source
AI summary
Methods and systems are provided for implementing training of learning models, including obtaining a pre-trained weight set for a learning model on a sample dataset and on a first loss function; selecting at least two tasks having heterogeneous features to be computed by a reference model; obtaining a reference dataset for the at least two tasks; designating a second loss function for feature embedding between the heterogeneous features of the at least two tasks; training the learning model on the first loss function and training the reference model on the second loss function, in turn; and updating the weight set based on a feature embedding learned by the learning model and a feature embedding learned by the reference model, in turn. Methods and systems of the present disclosure may alleviate computational overhead incurred by executing the learning model and loading different weight sets at a central network of the model.