Ephemeral Computing Nodes for Scalable Relational Data Processing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Relational databases are inefficient for handling large volumes of data and frequent additions, leading to high costs and limited scalability due to the use of persistent computing nodes, which are not easily spun down and do not enable horizontal scaling.

Innovation Solution

Implementing ephemeral computing nodes that spin up and down as needed, importing relational data from a data lake into a relational database management system (RDBMS) for processing, and then spinning down after completion, allowing for horizontal scalability and reducing costs by only using resources when active.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If persistent computing nodes are used to run relational databases, then data processing reliability is maintained, but computing resource costs increase and scalability is limited

Engineering Contradiction:

Improvedata processing reliabilityVSAvoidcomputing resource efficiency

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent applies the dynamics principle by transitioning from static persistent computing nodes to dynamic ephemeral computing nodes. The ephemeral nodes are dynamically provisioned only when computing tasks are present and automatically terminated after task completion. This dynamic approach maintains data processing reliability through orchestration-managed node lifecycle while dramatically improving computing resource efficiency by eliminating idle persistent nodes.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent implements the disposable objects principle by using ephemeral computing nodes that are created on-demand for specific computing tasks and then discarded. These short-living nodes replace expensive persistent computing nodes, providing the same functional capability only when needed. The orchestration system manages the creation and termination of these disposable nodes, ensuring reliable task execution while reducing overall computing resource costs.

Inventive Principle:

Principle #27Cheap short-living objects (Disposable)

2Duration of action of stationary object

If persistent computing nodes are used, then continuous data processing is enabled, but horizontal scaling capability is reduced

Engineering Contradiction:

Improvecontinuous data processingVSAvoidhorizontal scaling capability

Core Design Contradiction:

Duration of action of stationary objectVSAdaptability or versatility

Solution Approach 1:

The patent applies universality by designing ephemeral computing nodes that can be universally instantiated across multiple computing devices. The orchestration system can spin up identical node configurations on different devices as needed, enabling horizontal scaling. Each ephemeral node performs the same relational database processing function, allowing the system to scale out by adding more devices without being constrained by persistent node limitations.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Adaptability or versatility

If relational data is stored in a data lake, then storage flexibility is improved, but query processing performance deteriorates

Engineering Contradiction:

Improvestorage flexibilityVSAvoidquery processing speed

Core Design Contradiction:

Adaptability or versatilityVSSpeed

Solution Approach 1:

The patent implements the intermediary principle by introducing ephemeral computing nodes with embedded RDBMS as a mediator between the data lake and query processing operations. The data lake maintains flexible storage of relational data, while the ephemeral nodes temporarily load data into RDBMS structures optimized for querying. This intermediary layer preserves storage flexibility while dramatically improving query processing speed through appropriate data structure optimization.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent applies preliminary action by pre-loading relational data from the data lake into the RDBMS on ephemeral computing nodes before query processing begins. This preliminary data preparation and structuring in memory-optimized formats eliminates the performance penalty of direct data lake querying, while the ephemeral nature of the nodes ensures resources are freed after use.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11954531B2Use of relational databases in ephemeral computing nodes

Publication Date: 2024.04.09 OPTUM INC

AI summary

A method comprises: storing, by a computing system, a relational data in a data lake; spinning-up, by an orchestration system of the computing system, an ephemeral computing node on a computing device of the computing system; importing, by the ephemeral computing node, a copy of the relational data from the data lake into a relational database management system (RDBMS) installed on the ephemeral computing node; after importing the copy of the relational data into the RDBMS, performing, by the ephemeral computing node, a computing process that runs within the RDBMS and that uses the copy of the relational data; and after completion of the computing process on the ephemeral computing node, spinning-down, by the orchestration system, the ephemeral computing node.