In-place Database Update via Column-Oriented Delta Streams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Updating databases on low-end devices with limited resources is challenging due to large database sizes and limited networking and storage capabilities, especially when transmitting and processing full database updates.
Innovation Solution
Generating delta instructions by comparing column-oriented serialized streams of a source and target database to identify changes, allowing for in-place updates without creating a full copy of the target database, thus reducing the size of the update data and resource requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full database updates are transmitted to low-end devices, then the target database can be updated completely, but the network bandwidth and storage resources are overwhelmed
Solution Approach 1:
The patent extracts only the changed data items (deltas) from the target database and transmits them to the source database. Instead of transmitting the entire updated database, only the minimal necessary changes are extracted and transferred, dramatically reducing network bandwidth and storage requirements while maintaining update completeness.
Solution Approach 2:
The patent segments the database update process into two distinct phases: (1) generating delta instructions at the source database by comparing source and target databases, and (2) applying these delta instructions at the target database. This segmentation allows the heavy computation to occur at the source while minimizing data transmission to the resource-constrained target device.
2Quantity of substance
If column-oriented comparison is used to generate delta instructions, then the update data size is minimized, but the processing complexity increases
Solution Approach 1:
The patent changes the parameter of data organization from traditional row-oriented storage to column-oriented storage for the purpose of comparison. By organizing data columns vertically and serializing them, the patent enables more efficient delta detection that produces smaller update instructions, as identical column segments can be detected and compressed more effectively.
3Quantity of substance
If the source database is updated in-place using delta instructions, then storage resources are conserved, but the risk of update failures increases
Solution Approach 1:
The patent applies beforehand cushioning by generating and transmitting delta instructions that can be applied incrementally to the source database. This allows the update process to proceed in small, manageable steps rather than requiring a complete replacement, providing natural checkpoints and reducing the impact of potential failures while maintaining storage efficiency through in-place updates.
Data Source
AI summary
A method of generating a delta instructions record for updating a source database in-place, comprising obtaining a plurality of source data items contained in a plurality of cells of a plurality of tables organized in tuples and columns, creating a column oriented source stream serializing the source data items by concatenating a source data item of each cell of each column to a preceding source data item of a cell preceding the respective cell in the respective column, obtaining a plurality of target data items contained tables of a target database comprising data item(s) changed compared to the source database, creating a column oriented target stream serializing the plurality of target data items, delta instructions for applying changes detected between the source database and the target database by comparing between the source stream and the target stream, and outputting the delta instructions record to device(s) for updating their source database.