Atomic Full Data Update in Key-Value Databases
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing KeyValue databases lack support for full data updating, leading to inefficient and non-atomic processes when updating all data in a table, which affects the quality of data reading services.
Innovation Solution
A method and apparatus for generating and importing full data updating and deleting files in a KeyValue database, where each file includes rows with corresponding row keys and timestamps, ensuring atomicity and faster updating speeds by importing all data at once.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If table data is deleted and imported piece by piece, then the updating process can be completed, but the updating process takes a long time and lacks atomicity
Solution Approach 1:
The patent segments the full data updating process into two distinct files: a full data deleting file containing delete operations for all old data, and a full data updating file containing insert operations for all new data. This segmentation allows the system to process and import these files atomically as complete units, rather than processing individual rows sequentially, thereby maintaining atomicity while improving updating speed through batch processing.
2Ease of operation
If full data updating is performed by deleting and importing data piece by piece, then data can be updated, but the quality of data reading service deteriorates due to prolonged updating time
Solution Approach 1:
The patent performs preliminary actions by pre-generating the full data deleting file and full data updating file before the actual import operation. These files are prepared in advance with all necessary delete and insert operations, allowing the importing process to execute quickly as a single atomic operation. This preliminary preparation significantly reduces the actual updating time and minimizes service disruption, thereby improving data reading service quality.
3Reliability
If piece-by-piece data updating is used, then existing KeyValue database operations can be maintained, but the process lacks atomicity and affects service continuity
Solution Approach 1:
The patent merges all delete operations into a single full data deleting file and all insert operations into a single full data updating file. By combining multiple individual operations into consolidated files that can be imported atomically, the system achieves both atomicity (through single-file imports) and improved updating efficiency (through batch processing). This merging approach eliminates the need for sequential row-by-row operations while ensuring data consistency.
Data Source
AI summary
A method for updating a data table in a KeyValue database is provided. Raw data is received and formatted into a data updating file. An update timestamp is set for each column qualifier in each row of the data updating file. A data deleting file is generated. A delete timestamp is set for each row of the data deleting file. Each row of the data deleting file corresponds to one row of old data in the data table. Each row of the data deleting file includes a row key that is the same as the row key of the corresponding row of old data in the data table. The data updating file and the data deleting file are both imported into the data table. Consequently, timestamp is used to indicate the valid data in the data table. An apparatus configured to perform the method is also provided.