Dynamic Cache Placement in Dataflow Execution

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In dataflow execution environments, identifying datasets to cache dynamically during real-time executions is challenging, leading to increased costs due to multiple re-executions of operations, especially in complex dataflows with loops and iterations, where manual caching decisions are cumbersome and often suboptimal.

Innovation Solution

A dynamic cache placement method that adjusts cache placement plans based on real-time execution costs and updates the cost model during dataflow execution, using a cache placement algorithm to determine which datasets to cache and when to remove them from memory, ensuring optimal memory usage and reducing execution times.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of energy

If caching operations are used to avoid re-execution, then execution cost is reduced, but identifying which datasets to cache dynamically during real-time execution becomes complex and challenging

Engineering Contradiction:

Improveexecution costVSAvoidcomplexity of identifying datasets to cache

Core Design Contradiction:

Loss of energyVSDevice complexity

Solution Approach 1:

The system pre-calculates and stores cost models for all operations before execution. These cost models include estimated execution costs, data sizes, and other metrics needed for cache placement decisions. By preparing this information in advance, the system avoids complex real-time analysis during execution, resolving the contradiction between reducing execution cost and simplifying the complexity of identifying datasets to cache.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system automatically monitors execution progress, updates reference counts, and determines optimal cache placement decisions without manual intervention. The cost model and execution engine work together to self-adjust cache strategies based on real-time execution state, eliminating the need for complex manual analysis and reducing execution costs simultaneously.

Inventive Principle:

Principle #25Self-service

2Ease of manufacture

If manual cache placement is used, then implementation is simple, but cache placement decisions are suboptimal and increase execution costs

Engineering Contradiction:

Improveease of implementationVSAvoidexecution cost

Core Design Contradiction:

Ease of manufactureVSLoss of energy

Solution Approach 1:

The system continuously monitors execution progress and updates the cache placement strategy based on real-time feedback from the execution engine. As operations execute and reference counts update, the system recalculates optimal cache placement using the pre-computed cost model. This automated feedback loop eliminates manual intervention while achieving optimal cache placement, resolving the contradiction between ease of implementation and execution cost efficiency.

Inventive Principle:

Principle #23Feedback

3Ease of operation

If static cache placement plans are used, then implementation is straightforward, but they cannot adapt to changing execution states and increase re-execution costs

Engineering Contradiction:

Improveease of implementationVSAvoidexecution efficiency

Core Design Contradiction:

Ease of operationVSProductivity

Solution Approach 1:

The system transitions from static cache placement to dynamic cache placement by continuously monitoring execution state changes. As operations execute and reference counts change, the system dynamically updates the cache placement plan based on the cost model and current execution progress. This dynamic adaptation maintains ease of operation through automation while significantly improving execution efficiency by avoiding unnecessary re-executions.

Inventive Principle:

Principle #15Dynamics

4Productivity

If multiple actions are used in a dataflow, then processing capability is enhanced, but execution cost increases due to multiple re-executions of precedent operations

Engineering Contradiction:

Improveprocessing capabilityVSAvoidexecution cost

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The system applies cache operations selectively to specific datasets based on their individual characteristics and usage patterns in the dataflow. Rather than uniformly caching all intermediate results, the cost model identifies which specific operations and datasets benefit most from caching. This localized cache placement reduces execution cost for multi-action dataflows while maintaining their enhanced processing capability.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS10802975B2In-memory dataflow execution with dynamic placement of cache operations

Publication Date: 2020.10.13 EMC IP HLDG CO LLC

AI summary

A dataflow execution environment is provided with dynamic placement of cache operations. An exemplary method comprises: obtaining a first cache placement plan for a dataflow comprised of multiple operations; executing operations of the dataflow and updating a number of references to the executed operations to reflect remaining executions of the executed operations; determining a current cache gain by updating an estimated reduction in the total execution cost for the dataflow of the first cache placement plan; determining an alternative cache placement plan for the dataflow following the execution; and implementing the alternative cache placement plan based on a predefined threshold criteria. A cost model is optionally updated for the executed operations using an actual execution time of the executed operations. A cached dataset can be removed from memory based on the number of references to the operations that generated the cached datasets.