Vertex-Centric Expression Engine for Datacenter Telemetry
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current datacenter management systems face challenges in continuously monitoring the state of datacenters and their applications in near-real time, particularly in processing large quantities of metrics from storage volumes, which requires efficient and scalable solutions to handle dynamic changes and telemetry data streams.
Innovation Solution
A system utilizing vertex-centric programming and a scalable expression engine processes a domain model of a data storage system, generating a semantic model from topology information and updating expressions in near-real time, optimizing message propagation and processing to handle large data volumes efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a scalable expression engine using vertex-centric programming is implemented, then the system can scale to handle large datacenters and process telemetry data streams efficiently, but the device complexity and implementation difficulty increase
Solution Approach 1:
The system segments the datacenter topology into a graph structure where managed objects are represented as vertices and relationships as edges. This segmentation allows the expression engine to process telemetry data in distributed, parallel fashion across multiple vertices, enabling scalability to large datacenters while maintaining manageable complexity through modular processing units.
Solution Approach 2:
The patent introduces a new dimensional framework by mapping expression evaluation onto a vertex-centric programming model that operates across the graph dimension of the topology. This dimensional transformation enables the system to handle complex expressions involving multiple managed objects by evaluating them locally at each vertex and propagating results through the graph, rather than using centralized processing.
2Measurement precision
If continuous near-real-time monitoring of datacenter state is implemented, then monitoring accuracy and responsiveness improve, but the computational overhead and processing time increase
Solution Approach 1:
The system performs preliminary actions by pre-compiling expressions into executable form and pre-establishing the graph topology structure before telemetry data arrives. This allows the expression engine to immediately evaluate expressions against incoming telemetry data without compilation overhead, achieving near-real-time monitoring with minimal processing delay.
Solution Approach 2:
The patent implements continuous evaluation of expressions against the streaming telemetry data, maintaining an always-updated state of the datacenter. The vertex-centric model allows expressions to be continuously evaluated as new telemetry arrives at relevant vertices, ensuring monitoring accuracy without requiring periodic batch processing that would introduce delays.
3Reliability
If expressions are continuously evaluated against changing graph and telemetry data streams, then the current state of the datacenter is accurately determined, but the computational resources and processing load increase
Solution Approach 1:
The vertex-centric expression engine enables each vertex to autonomously evaluate expressions local to that vertex using telemetry data relevant to its managed objects. This self-service approach distributes computational load across the entire graph rather than concentrating it in a central processor, reducing overall computational resource requirements while maintaining accurate state determination through distributed evaluation.
Data Source
AI summary
Methods and apparatus for distributed evaluation of expressions defined by a domain model including classes with attributes and relationships among the classes. Messages containing attribute updates are sent and processed using vertex-centric graph processing technique to evaluate expressions and determine a state of the system.