Database Table Change Detection via Timestamps and Hashes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Consuming applications face challenges in determining which rows in schema-less database tables have been added, updated, or deleted during bulk load processes without complete before and after snapshots or additional tools.
Innovation Solution
The method involves loading records with timestamps and hash values into a metadata table, allowing consuming applications to identify records based on their loading time periods and hash values to determine additions, deletions, and updates without requiring complete snapshots or additional tools.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If complete before and after snapshots are used to determine database changes, then measurement precision is improved, but loss of time and use of energy worsen due to the overhead of capturing and comparing complete snapshots
Solution Approach 1:
The patent extracts only the essential change information (timestamps and hash values) from complete database snapshots, storing them in a metadata table. This allows consuming applications to determine additions, deletions, and updates by comparing metadata records rather than comparing complete before and after snapshots, thereby reducing time overhead while maintaining measurement precision.
2Measurement precision
If additional tools are used to track database changes, then measurement precision is improved, but device complexity worsens
Solution Approach 1:
The patent merges the change tracking functionality into the existing database metadata table, combining storage of record data with storage of change tracking information (timestamps and hash values). This integration eliminates the need for separate additional tools while maintaining accurate change detection capabilities, thereby reducing device complexity.
3Adaptability or versatility
If schema-less database tables are used to provide flexibility, then adaptability is improved, but difficulty of detecting and measuring changes worsens
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing hash values and timestamps for each record when it is loaded into the database. This preliminary preparation of metadata information enables consuming applications to easily detect and measure changes by comparing these pre-stored values, thereby reducing the difficulty of change detection while maintaining schema-less flexibility.
Data Source
AI summary
A method, system and computer program product for determining changes to database tables. In an embodiment, the method comprises loading a first group of records in a first time period in a database; loading a second group of records in a second time period in the database; selecting one or more of the records in the database; and for each of the selected records, determining whether it was loaded into the database during the first time period or the second time period, and providing the each record with a specified designation based on the time period during which the each record was loaded into the database. In an embodiment, identified times when the records were loaded in the database are used to determine the time period during which the each record was loaded in the database.