Hierarchical Event Queues for Serverless Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Applications requiring sequential invocations of multiple functions in serverless computing models experience increased delays due to functions triggering each other and sharing/application data through a global event queue, limiting the development of low-latency applications.
Innovation Solution
A method and host configuration that utilize hierarchical queues to categorize hosts based on geographical or logical criteria, allowing events to be processed locally and reducing latency by creating shortcuts for subsequent event processing, with backup mechanisms to ensure reliability in case of host failure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If functions trigger each other through a global event queue in serverless computing, then applications can be developed using simple event-driven programming, but latency increases due to sequential processing through multiple queues
Solution Approach 1:
The patent segments the global event queue into hierarchical levels (level 0 being the local queue at the function execution host, and higher levels being remote queues). This segmentation allows events to be processed locally when possible, reducing latency, while maintaining the global queue structure for events that need remote processing. The segmentation resolves the contradiction by enabling fast local processing for simple cases while preserving the unified event-driven model for complex scenarios.
Solution Approach 2:
The patent introduces level 0 queues as intermediary structures between the global event queue and function execution hosts. These local queues act as buffers that can hold events locally at the host, allowing functions to be triggered immediately without waiting for remote queue processing. This intermediary mechanism reduces latency while maintaining the overall event-driven architecture.
2Reliability
If events are processed through remote hosts via higher level queues, then data locality can be optimized, but processing delays increase due to inter-host communication
Solution Approach 1:
The patent implements local quality by allowing each host to have its own level 0 queue that prioritizes local event processing. When a function is deployed to a host, events related to that function can be processed locally through the level 0 queue, eliminating the need for remote communication. This local processing approach maintains data locality benefits while avoiding the latency of inter-host communication for local events.
Solution Approach 2:
The patent enables preliminary action by pre-positioning events in level 0 queues at the appropriate hosts before function execution is needed. Events can be queued locally in advance, and when the target function is ready, processing can begin immediately without waiting for remote queue retrieval. This preliminary positioning of events resolves the contradiction by preparing data locally ahead of time.
3Device complexity
If a unified global event queue is used across all hosts, then system simplicity is maintained, but latency increases for sequential function invocations
Solution Approach 1:
The patent applies the nested doll principle by creating a hierarchical queue structure where level 0 queues (local to each host) are nested within the broader global event queue system. The level 0 queues are essentially local instances of the global queue concept, providing a simplified interface for local processing while maintaining integration with the overall distributed system. This nesting resolves the contradiction by providing simple local queues that are part of the unified global system.
Solution Approach 2:
The patent adds a new dimension to the event queue system by introducing hierarchical levels (level 0, level 1, etc.) rather than using a single flat global queue. This dimensional change allows events to be routed to appropriate levels based on their processing requirements, with level 0 providing fast local processing and higher levels providing remote processing capabilities. This multi-dimensional structure resolves the contradiction by providing both simplicity and performance.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention discloses a method of managing events in a network that adopts event-driven programming framework, the network comprising: a plurality of physically distributed hosts, each of which is configured to process at least one event in the network; at least a first entity, configured to maintain a first queue comprising a first plurality of events to be processed by at least one host belonging to a first group of host(s), and at least a second entity, configured to maintain a second queue comprising a second plurality of events to be processed by at least one host belonging to a second group of host(s), wherein, the first group of host(s) comprises the second group of host(s); the method comprising steps of: a) determining a first event in the first queue as an event that is to be processed; b) determining whether the first event satisfies a first condition, the first condition indicating that the first event can be processed by at least a first host belonging to the second group of host(s); b1) if so, retrieving the first event from the first queue to the second queue, c) processing the first event on the first host, d) receiving a report of a second event that is newly generated; e) determining whether the second event satisfies a second condition, the second condition indicating that the second event can be processed by at least a second host belonging to the second group of host(s), e1) placing the second event to the second queue and then processing the second event the at the second host, if the second event satisfies the second condition; e2) reporting the second event to the first entity maintaining the first queue, if the second event does not satisfy the second condition.