Graph Engine Transparent Remote Access via REST
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graph analytic systems require a time-consuming three-step process for evaluating and optimizing graph analysis, involving a laptop, server-class machine, and distributed cluster, due to the need for multiple configurations and high computational costs, making it inefficient to determine the best analysis method before using expensive server-class machines.
Innovation Solution
A graph engine with multiple configuration modes (embedded, server, and distributed) that decouples analysis request execution, allowing for transparent remote access using REST, and adapts to different deployment scenarios, facilitating efficient workload management and remoting through mechanisms like TCP and HTTP, enabling efficient processing and result delivery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If graph analysis is performed on a server-class machine with full configuration, then computation speed and performance are improved, but setup time and configuration complexity increase
Solution Approach 1:
The graph analytic engine is designed to operate in multiple deployment configurations (embedded mode, server mode, distributed mode) using the same core codebase. This allows the system to provide universal graph analysis functionality across different hardware platforms without requiring separate configurations for each deployment scenario, thereby reducing setup complexity while maintaining computation speed on server-class machines.
Solution Approach 2:
The system dynamically adapts to different deployment configurations through configuration modes that are activated based on the runtime environment. The engine can automatically detect and adjust to embedded, server, or distributed modes, making the configuration process dynamic rather than static, which reduces the burden of manual setup while preserving high-performance capabilities when deployed on server-class machines.
2Measurement precision
If graph analysis variations are tested on a big dataset, then accurate performance comparison is achieved, but computation time and energy consumption increase
Solution Approach 1:
The system enables testing of graph analysis variations on partial datasets or sampled subsets of the full big dataset. By performing initial evaluations on representative samples, developers can accurately compare performance characteristics without incurring the full computational cost of processing the entire large dataset, thereby reducing computation time and energy consumption while maintaining sufficient measurement precision for decision-making.
3Adaptability or versatility
If graph analysis is distributed across a cluster, then scalability is improved, but system complexity and deployment difficulty increase
Solution Approach 1:
The graph analytic engine is segmented into modular components that can be independently deployed across cluster nodes. Each node runs an instance of the engine that can process partitioned graph data, and the system provides automated mechanisms for data distribution, task coordination, and result aggregation. This segmentation enables scalability across distributed clusters while the automation of deployment tasks reduces the complexity burden on users.
4Ease of operation
If multiple analysis variations are tested before production, then optimal analysis selection is achieved, but time and computational resources are consumed
Solution Approach 1:
The system enables rapid prototyping and testing of multiple graph analysis variations using lightweight, disposable test configurations that can be quickly created, executed, and discarded. By allowing developers to spin up temporary analysis jobs with different parameters and algorithms on demand, the system facilitates easy comparison of variations without committing significant computational resources to each test, thereby improving development efficiency while maintaining the ability to select optimal analyses.
Data Source
AI summary
Techniques herein decouple available results, from graph analysis execution, to adapt to various deployment configurations. In an embodiment, a graph engine is deployed that has multiple mutually-exclusive configuration modes that include being embedded within a software application, centrally serving software applications, or distributed amongst a cluster of computers. Based on a current configuration mode of the graph engine, a software application receives or generates an analysis request to process a graph. The software application provides the analysis request to the graph engine in exchange for access to a computational future, of the graph engine, that is based on the analysis request and the graph. Based on a proxy of said computational future, the software application accesses a result of the analysis request. In an embodiment, a remote proxy exchanges representational state transfer (REST) messages. Network mechanisms, such as transport control protocol (TCP) and hypertext transfer protocol (HTTP), provide enhanced remoting.