Log Mining Read Consistency via Data Image Merging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Log mining systems face inconsistencies when retrieving data from transaction logs and databases, particularly with non-logged, partially logged, encrypted, or compressed data, leading to inconsistencies across transaction boundaries.
Innovation Solution
A computer-readable storage medium with executable instructions that retrieves data changes from both transaction logs and databases, using a data image to ensure read consistency by accessing data as of a specific Logical Change Number (LCN), merging fetched data with change data records to create a consistent composite record.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is not fully logged or encrypted for security or performance reasons, then log writing performance is improved and overhead is reduced, but data consistency between transaction log and database deteriorates
Solution Approach 1:
The patent introduces data images as an intermediary mechanism between the transaction log and the database. Data images capture complete data states at specific logical change numbers (LCNs), serving as a mediator that allows the log mining system to retrieve complete data even when the transaction log contains only partial or encrypted information. This resolves the consistency issue without requiring full logging of all data changes.
Solution Approach 2:
The patent creates copies of complete data states in the form of data images at specific transaction boundaries. These copies preserve the full data content without requiring the transaction log to store complete information. The log mining system can then use these copies to supplement partial or encrypted log data, maintaining consistency while preserving logging performance benefits.
2Loss of information
If data is retrieved from both transaction log and database to supplement missing information, then complete data retrieval is achieved, but inconsistencies between log and database deteriorate
Solution Approach 1:
The patent performs preliminary actions by creating data images at specific logical change numbers before log mining operations occur. These pre-captured data states contain complete information that can be used during log mining to ensure consistency. By preparing the data images in advance, the system avoids the need to query the current database state during log mining, which could introduce inconsistencies.
Solution Approach 2:
The patent replaces the mechanical approach of querying the live database during log mining with a substitution approach using pre-captured data images. Instead of dynamically accessing the database (which could return inconsistent data), the system substitutes this with static data images that capture complete data states at specific transaction boundaries, ensuring consistency without requiring ongoing database access.
3Reliability
If all data changes are fully logged, then data consistency is maintained, but log writing overhead and commit time latency increase
Solution Approach 1:
The patent applies partial action by logging only the necessary changes in the transaction log (such as change vectors or differential data) rather than complete data rows. This reduces the volume of data that must be written to the log, decreasing commit time latency. The missing complete data is then retrieved from data images during log mining operations, achieving consistency without the overhead of full logging.
Data Source
AI summary
A computer readable storage medium includes executable instructions to receive a specification for requested data in a log based replication system where data changes are retrieved from both a transaction log and a database. It is determined that the requested data is compromised. A search for a data image containing the requested data is initiated. The data image is accessed to secure the requested data. The requested data is returned.