Intermediate Metrics Aggregation Service for Serverless Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In serverless event-driven computing environments, gathering metrics from ephemeral and high-volume instances is challenging due to latency issues and overwhelming of centralized logging services, as functions trigger, execute, and retire quickly, making it difficult to collect and log metrics efficiently.
Innovation Solution
An intermediate metric aggregation service processes metrics reports from a fleet of containerized execution environments, using asynchronous logging and batching to aggregate and transmit metrics, allowing for efficient logging and reporting even when functions terminate without saving execution state, thereby reducing latency and managing high volumes of concurrent connections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If synchronous metric logging is used in serverless environments, then metrics can be collected accurately, but function execution latency increases significantly
Solution Approach 1:
The patent introduces an intermediate logging service that acts as a mediator between the serverless function and the centralized logging service. The function writes metrics to this intermediate service asynchronously, which then batches and forwards them to the centralized logging service. This intermediary layer decouples the metric collection from the function execution, eliminating latency while ensuring metrics are captured.
Solution Approach 2:
The intermediate logging service pre-processes and batches metrics before forwarding them to the centralized logging service. By performing preliminary aggregation and buffering of metric data, the system reduces the immediate burden on the centralized service and allows functions to complete execution without waiting for metric confirmation.
2Quantity of substance
If centralized logging service collects metrics from all individual function executions, then comprehensive metrics are gathered, but the logging service becomes overwhelmed by high volume of concurrent connections
Solution Approach 1:
The patent segments the logging function by introducing an intermediate logging service that handles individual function metric submissions separately from the centralized logging service. This segmentation distributes the connection handling burden, allowing the centralized service to process aggregated data while the intermediate service manages high-volume concurrent connections from numerous short-lived functions.
Solution Approach 2:
The intermediate logging service merges multiple individual metric submissions into batches before forwarding to the centralized logging service. By combining multiple small metric events into larger batch operations, the system reduces the total number of connections and messages the centralized service must handle, preventing overload while maintaining comprehensive metrics collection.
3Loss of information
If metrics are logged synchronously before function retirement, then complete metric data is captured, but function execution time increases
Solution Approach 1:
The intermediate logging service enables asynchronous metric capture by providing a fire-and-forget interface to functions. Metrics are written to the intermediary without blocking function completion, ensuring no metric data is lost while maintaining short execution times. The intermediary ensures durability through batching and confirmation mechanisms.
Data Source
AI summary
Devices and processes perform a metrics aggregation process within the context of a serverless compute environment that includes a fleet of execution environment containers for executing ephemeral functions. In response to receipt of triggering events, functions, registered to handle the events, are launched in separate execution environments of an event-driven compute service. The functions generate metrics that are sent to the execution environment via a call (e.g., via asynchronous call) to the execution environment, and the functions may retire, without saving execution state for the function, and before the execution environment passes the metrics to an intermediate metrics aggregation service. The metric aggregation system receives the metrics generated by the fleet of instances of the function from the plurality of execution environments of the event-driven compute service, batch processes the metrics into an aggregated record, and transmits the aggregate record to a metrics reporting service.