Service Mesh Proxy Snoop Pre-Instantiation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In Serverless computing with a service mesh environment, the latency and delay associated with instantiating and executing multiple function codes in a sequence are significant due to the need for individual client requests to spawn each code segment, leading to operational inefficiencies and increased response times.
Innovation Solution
Implementing a data plane triggered method where a network proxy, such as Envoy, snoops the return traffic to identify and instantiate subsequent function codes based on Uniform Resource Identifiers, allowing them to be ready for execution before receiving the corresponding client request, thereby reducing the delay and improving response times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If individual client requests are used to spawn each function code segment in sequence, then service mesh architecture maintains simplicity and reliability, but latency and response time increase significantly
Solution Approach 1:
The system performs preliminary actions by snooping response traffic to identify upcoming function codes and instantiating them before they are actually needed. This advance preparation eliminates cold-start latency when client requests arrive, as the function codes are already initialized and ready to execute immediately.
Solution Approach 2:
A network proxy acts as an intermediary between the service mesh and function code execution environment. The proxy intercepts and snoops response traffic, extracts function code identifiers, and triggers pre-instantiation without requiring direct client involvement. This intermediary mechanism decouples the timing of function code preparation from client request timing.
2Speed
If function codes are pre-instantiated based on snoop traffic, then response time decreases, but system complexity increases due to additional traffic analysis and triggering mechanisms
Solution Approach 1:
The network proxy performs multiple functions: it routes normal traffic, snoops response traffic for function code identification, and triggers pre-instantiation of function codes. By consolidating these diverse functions into a single component, the system avoids the complexity of separate specialized systems for each function.
Solution Approach 2:
The system uses its own response traffic as the trigger signal for pre-instantiating function codes. The response traffic naturally contains information about upcoming function calls, and the network proxy leverages this existing information flow to automatically initiate pre-instantiation without requiring external control systems or additional signaling mechanisms.
3Use of energy by moving object
If multiple function codes are instantiated in sequence waiting for client requests, then resource allocation remains efficient, but productivity and throughput decrease
Solution Approach 1:
Function codes are instantiated in advance based on predicted usage patterns derived from snoop traffic analysis, rather than waiting for actual client requests. This preliminary action ensures that when multiple function codes need to execute in sequence, they are already prepared and can run concurrently or immediately without waiting for individual request processing to complete.
Solution Approach 2:
The system maintains continuous useful action by keeping function codes in a ready state through pre-instantiation. Instead of alternating between idle waiting for requests and active execution, the system maintains a continuous state of readiness for multiple function codes, enabling seamless processing and improving overall throughput without significant additional resource consumption.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present technology is directed to a system and method for automatic triggering of relevant code segments corresponding to a sequence of code segments or function codes having a preferred execution order. The automatic triggering action is based on the snooping of a response generated from an execution of a previous code segment. Information with respect to the next code segment in the preferred execution order may be obtained by directing a network proxy, such as Envoy to snoop the Uniform Resource Identifier (URI) field of a response packet being forwarded to a client entity. In this way, a network proxy may preemptively spawn and instantiate the following function codes (pointed to by the snooped Uniform Resource Identifier) prior to receiving the corresponding client request. As such, by the time a client request for the subsequent function code is received the code ready for execution.