Database Modification Using Script Component to Reduce ETL Overhead

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional extract-transform-load (ETL) pipelines are inefficient due to the use of time-intensive operations such as authentication and outer join operations, which consume significant computing resources and network bandwidth.

Innovation Solution

The method involves executing a script component with one or more functions to transform a database, allowing for the modification of a database defined from an extant hypercube without modifying the original hypercube, thus avoiding time-intensive operations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If conventional ETL pipelines are used to modify databases, then data transformation can be achieved, but computing resources and network bandwidth are consumed excessively due to authentication and outer join operations

Engineering Contradiction:

Improvedatabase modification efficiencyVSAvoidcomputing resource usage

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The patent extracts and eliminates the time-intensive authentication and outer join operations from the database modification process. By using a script component that directly transforms the database without these conventional ETL steps, the system removes the harmful operations while preserving the essential data transformation functionality.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Instead of following the conventional ETL approach of extract-transform-load with authentication and joins, the patent inverts the process by using a script component that directly modifies the database structure and data without these intermediate steps. This reversal eliminates the computational overhead while achieving the same transformation goals.

Inventive Principle:

Principle #13The other way round (Inversion)

2Productivity

If conventional ETL pipelines are used to modify databases, then data transformation can be achieved, but network bandwidth is consumed excessively due to authentication and outer join operations

Engineering Contradiction:

Improvedatabase modification efficiencyVSAvoidnetwork bandwidth consumption

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The patent extracts and eliminates the network-intensive authentication and outer join operations from the database modification process. By using a script component that operates directly on the database without these conventional ETL steps, the system removes the network overhead while preserving the essential data transformation functionality.

Inventive Principle:

Principle #2Taking out (Extraction)

3Reliability

If authentication and outer join operations are performed in ETL pipelines, then data integrity can be ensured, but processing time increases significantly

Engineering Contradiction:

Improvedata integrityVSAvoidprocessing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent performs data integrity checks and transformations as preliminary actions within the script component itself, before the actual database modification occurs. This allows the system to ensure data integrity without requiring separate authentication and join operations during the execution phase, thereby reducing processing time.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20250181558A1Database modification using a script component

Publication Date: 2025.06.05 QLIK TECH INTERNATIONAL AB

AI summary

Technologies are provided for database modification using a script component. In some embodiments, a method comprises executing, by a computing system comprising at least one processor, a script component comprising one or more functions to transform a database. The method also includes modifying, by the computing system, the database based on at least one transformation defined by the one or more functions in response to executing the script component.