ETL Graph Preview Caching for Subgraph Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing process flow design tools consume significant processing resources due to inefficiencies in re-executing portions of process flows when users interact with graphs representing process flows, as cached results cannot be easily applied to subsequent subgraphs that differ from the cached ones.
Innovation Solution
Implementing a system that caches execution results associated with selected subgraphs and applies these cached results when a subsequently selected subgraph matches a cached subgraph, thereby skipping re-execution of computer-readable instructions for portions of the process flow.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If process flow execution previews are generated by re-executing portions of the process flow each time a user interacts with the graph, then the previews accurately reflect the current state of the process flow, but significant processing resources are consumed due to redundant executions
Solution Approach 1:
The system performs preliminary execution of process flow portions and stores the results in a cache before users interact with the graph. When a user requests a preview, the system checks the cache first and retrieves stored results if available, avoiding re-execution. This preliminary action ensures accuracy while reducing processing resource consumption by eliminating redundant executions.
Solution Approach 2:
The system creates copies of execution results from previously executed process flow portions and stores them in a cache. Instead of re-executing the same portions, the system retrieves and uses these cached copies when generating previews. This copying approach maintains accuracy by using actual execution results while significantly reducing processing resources by avoiding repeated executions of identical process flow segments.
2Use of energy by moving object
If the system caches execution results for subgraphs to reduce re-execution, then processing resource consumption is reduced, but cached results cannot be easily applied to subsequently selected subgraphs that differ from the cached ones
Solution Approach 1:
The system segments the process flow graph into distinct subgraphs and caches execution results at multiple hierarchical levels. When a user selects a subgraph, the system checks for cached results of that specific subgraph first, then checks for cached results of parent subgraphs that contain the selected subgraph. This segmentation approach allows cached results to be adaptably applied to different subgraphs by matching at various levels of the hierarchy, reducing processing resources while maintaining versatility.
Solution Approach 2:
The cache system is designed with multi-functionality to serve multiple subgraph selection scenarios. It can retrieve cached results for exact matches, partial matches, and hierarchical parent-child relationships between subgraphs. This universal cache mechanism allows a single cached result to serve multiple different subgraph selections, enhancing adaptability while reducing processing resource consumption through reusable cached data.
Data Source
AI summary
Disclosed are some implementations of systems, apparatus, methods and computer program products for executing a process flow represented by a graph or portion thereof using cached subgraphs. A first request to execute a first portion of a process flow is processed, where the first portion of the process flow is represented by a first subgraph of a graph representing the process flow and a final node of the first subgraph corresponds to a set of computer-readable instructions. The first portion of the process flow is executed such that a first output of executing the first portion of the process flow is obtained. The first subgraph is stored in association with the first output in a first cache entry of a cache. A second request to execute a second portion of the process flow is processed, where the second portion of the process flow is represented by a second subgraph of the graph. At least one cache entry for which a corresponding subgraph matches at least a portion of the second subgraph is identified in the cache, where the at least one cache entry includes the first cache entry. The first output is retrieved from the first cache entry, a node of the second subgraph to which the final node of the first subgraph is connected is identified, and the second portion of the process flow is executed by providing the first output as input to the identified node of the second subgraph without executing the set of computer-readable instructions.