Database File Mirroring via Transaction Log Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database architectures face inefficiencies in mirroring binary large object (blob) data changes across secondary database servers due to the lack of inclusion in database logs, often requiring the copying of entire files even when only a small part has changed, leading to excessive data transfer.
Innovation Solution
A method that intercepts file commands with filename, offset, and size information, writes these to a transaction log, and sends only the changed portions to secondary database servers, allowing for efficient mirroring of file data changes by identifying and transferring specific parts of files rather than entire files.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If entire files are copied to secondary servers even when only small parts have changed, then data consistency is maintained, but network data transfer volume increases significantly
Solution Approach 1:
The patent segments files into smaller blocks and tracks changes at the block level using offsets and sizes. Instead of copying entire files, only the specific blocks that have changed are identified and transferred to secondary servers. This segmentation allows selective replication of only the necessary data portions, resolving the contradiction between maintaining data consistency and reducing network transfer volume.
Solution Approach 2:
The patent extracts and logs only the changed portions of files (specific blocks identified by offset and size) rather than handling entire files. The transaction log captures precise change information, allowing the system to extract and transfer only the necessary data blocks to secondary servers, thereby maintaining consistency while minimizing network data transfer.
2Quantity of substance
If blob data changes are not included in database logs, then storage efficiency is improved, but the ability to mirror changes to secondary servers deteriorates
Solution Approach 1:
The patent introduces a file system layer as an intermediary between blob data storage and the database log system. This file system intercepts file commands, tracks changes to blob data through a separate mechanism, and makes this change information available to the database replication system. The intermediary enables change tracking without requiring blob data changes to be included directly in database logs, thus maintaining storage efficiency while restoring mirroring capability.
3Measurement precision
If file commands are intercepted and tracked with offset and size information, then precise change identification is achieved, but system complexity increases
Solution Approach 1:
The patent implements a universal file system layer that handles multiple functions: intercepting file commands, tracking changes with offset and size information, logging transaction details, and enabling replication. This multi-functional file system approach achieves precise change identification while consolidating complexity into a single manageable component rather than requiring separate mechanisms for each function.
Data Source
AI summary
High availability and disaster recovery can be achieved within a database management system by detecting which parts of a file have changed and sending the changed parts to secondary servers, without sending the entire file that has experienced the changes. A log cracker reads data from a transaction log that stores records from a database and a separately maintained file system. The log cracker sends mirror commands with some of the transaction log data to the file system to initiate mirroring of the data from the file system to secondary servers. The log data is then subsequently sent to the secondary servers as well, to enable identification of changed data without having to transfer the entire file that has been changed.