Data Warehouse Versioning Engine for Record Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data warehouse management systems lack an effective versioning scheme, making it difficult to track changes and perform data analysis over time, as they typically involve manual updates and replacement of previous versions, which are irretrievable.
Innovation Solution
A method and system that uses a data warehouse engine to manage data records with a fact table organized by primary key, involving steps like identifying existing records, assigning version numbers, setting up temporary tables, and bulk loading new data records with updated version numbers, allowing for tracking of record versions and updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual updates via low-level RDBMS operations are used, then the data warehouse can be updated, but the update process becomes inefficient and difficult to manage when the number of users and updates increase
Solution Approach 1:
The patent introduces a data warehouse engine as an intermediary layer between users and the underlying RDBMS. This engine provides automated ETL processes, versioning management, and bulk update capabilities, eliminating the need for manual low-level operations while maintaining data integrity and enabling efficient handling of multiple users and updates
Solution Approach 2:
The system implements self-service capabilities through automated data loading, transformation, and versioning processes. The data warehouse engine automatically manages update operations, version number assignment, and record tracking without requiring manual intervention, thereby improving productivity while reducing management complexity
2Productivity
If the previous version of data records is replaced with an updated file, then the data warehouse can be updated, but the ability to track changes or perform data analysis on data over time is lost
Solution Approach 1:
The patent segments the data storage structure by introducing version number fields and status indicators that divide each data record into distinct versioned components. This allows the system to maintain multiple versions of data records simultaneously, enabling both fast updates and historical tracking without replacing previous versions
Solution Approach 2:
The system changes the parameter structure of data records by adding version number parameters and current version status flags. These parameter changes enable the system to track transformations over time while maintaining update efficiency, as each record carries its version information as an inherent attribute
3Reliability
If versioning is implemented in the data warehouse, then change tracking and historical analysis become possible, but the data warehouse structure and update process become more complex
Solution Approach 1:
The patent implements a universal versioning framework that integrates change tracking, historical analysis, and update management into a single cohesive system. The data warehouse engine provides multi-functional capabilities including automated version number assignment, status tracking, and bulk update operations, reducing the need for separate complex structures for each function
Data Source
AI summary
A method, system, and computer program product are disclosed. Exemplary embodiments of the method, system, and computer program product may include hardware, process steps, and computer program instructions for supporting versioning in a data warehouse. The data warehouse may include a data warehouse engine for creating a data warehouse including a fact table and temporary tables. Updated or new data records may be transferred into the data warehouse and bulk loaded into the temporary tables. The updated or new data records may be evaluated for attributes matching existing data records. A version number may be assigned to data records and data records may be marked as being the most current version. Updated and new data records may be bulk loaded from the temporary tables into the fact table when a version number or a version status is calculated.