Intermediate Network Node With Imitation Model for Low-Latency Requests
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing communication networks face challenges in reducing execution latency for real-time applications due to the high computational requirements of computational graph models, especially in devices with limited resources, and existing solutions like proxy servers and edge computing are either expensive or limited by memory constraints.
Innovation Solution
Implementing an intermediate network node with an imitation model that is a simplified version of the computational graph model, allowing it to intercept requests, perform confidence checks, and either respond locally or relay requests to the executing node based on confidence thresholds, thereby reducing computational load and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If computing nodes are placed closer to the network edge to reduce execution latency, then response time is improved, but deployment cost increases significantly
Solution Approach 1:
The patent creates a simplified copy (imitation model) of the complex computational graph model and places it at the network edge. This copy handles common requests locally, providing fast responses without requiring expensive edge computing infrastructure. The imitation model is a lightweight version that reproduces the essential functionality of the full model for typical workloads.
Solution Approach 2:
The patent introduces an intermediate network node that acts as a mediator between requesting nodes and the central computing infrastructure. This intermediate node runs the imitation model and intercepts requests, deciding whether to handle them locally or forward to the full model. This intermediary layer reduces the need for expensive edge computing by filtering requests before they reach the edge.
2Loss of time
If proxy servers are used at the edge to cache responses, then response time is improved for frequent requests, but the solution is limited by memory constraints and specific request patterns
Solution Approach 1:
The patent changes the fundamental parameter of what is being cached - instead of caching raw responses (memory-intensive), it caches a trained imitation model (computation-intensive but memory-efficient). This parameter change allows the system to handle diverse request patterns without being constrained by limited edge memory, as the model can generalize to new requests rather than requiring exact matches.
Solution Approach 2:
The patent creates a functional copy of the model's decision-making capability rather than copying response data. The imitation model replicates the behavior of the full computational graph model, enabling the system to generate responses for unseen requests while using minimal memory compared to traditional caching approaches.
3Loss of time
If full computational graph models are executed at the network edge, then execution latency is reduced, but computational resource requirements increase
Solution Approach 1:
The patent extracts only the essential computational components needed for handling typical requests and places them at the edge in the form of an imitation model. The full computational graph model remains in the cloud, and only the necessary subset of functionality is replicated at the edge, significantly reducing computational resource requirements while maintaining low latency for common workloads.
Solution Approach 2:
The patent applies partial action by deploying a simplified imitation model rather than the full computational graph model at the edge. This partial deployment is sufficient to handle the majority of requests with acceptable accuracy, avoiding the excessive computational resource consumption that would result from running the complete model at the edge.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Embodiments herein relate, in some examples, to an intermediate network node (11) configured to operate in a communication network, wherein the communication network comprises a requesting node and an executing network node comprising a computational graph model. The intermediate network node (11) is configured with an imitation model, wherein the imitation model is a limited version of the computational graph model, and wherein the imitation model is a model requiring less computational resources to converge when compared to the computational graph model.