Standby Database Memory Pool Segmentation for Transactional Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Standby database systems face challenges in managing memory and resources to maintain transactional consistency with primary databases, leading to inefficiencies in processing power and storage, particularly due to varying transaction sizes and fluctuating workloads, which can result in memory thrashing and redundancy in buffered records.
Innovation Solution
The implementation of dedicated memory pools for transaction metadata, a garbage collection algorithm that reverses the allocation order of memory chunks, and a deduplication algorithm to reduce network traffic and processing costs, along with pre-processing transaction metadata and dividing work among apply slave processes to prevent blocking and optimize memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the standby database buffers transaction records to maintain transactional consistency, then data consistency is improved, but memory consumption increases and memory thrashing occurs due to varying transaction sizes
Solution Approach 1:
The patent segments memory management by introducing separate memory pools for different transaction metadata types (e.g., one pool for transaction records, another for change records). This segmentation allows the system to allocate memory more efficiently based on actual transaction size requirements, preventing memory thrashing while maintaining transactional consistency.
Solution Approach 2:
The patent implements dynamic memory allocation strategies where memory pools can be adjusted based on workload conditions. The garbage collection algorithm dynamically identifies and releases memory from inactive transaction records, allowing the system to adapt memory consumption to actual needs while preserving transactional integrity.
2Reliability
If the standby database buffers all transaction records, then transactional consistency is maintained, but processing efficiency decreases due to redundancy in buffered records
Solution Approach 1:
The patent extracts and removes redundant information from the buffered transaction records by implementing a deduplication mechanism. The system identifies and eliminates duplicate change records while maintaining only the essential transaction metadata needed for consistency, thereby improving processing efficiency without sacrificing data integrity.
Solution Approach 2:
The patent changes the parameters of stored transaction records by selectively preserving only the necessary fields and metadata. The system transforms the representation of transaction records to minimize redundancy while maintaining the information needed for transactional consistency, thus improving processing efficiency.
3Productivity
If the reference timestamp is advanced continuously, then query availability is improved, but transactional consistency may be compromised if not synchronized with primary database
Solution Approach 1:
The patent implements a feedback mechanism where the reference timestamp advancement is continuously monitored and adjusted based on synchronization status with the primary database. The system uses feedback from transaction processing to ensure that timestamp advancement maintains transactional consistency while maximizing query availability.
Solution Approach 2:
The patent performs preliminary synchronization actions before advancing the reference timestamp. The system pre-processes and validates transaction records to ensure consistency is maintained, then safely advances the timestamp to enable queries without compromising transactional integrity.
4Productivity
If multiple processes buffer change records simultaneously, then processing throughput is improved, but memory management complexity increases and fragmentation occurs
Solution Approach 1:
The patent segments the memory management responsibilities among multiple processes by assigning each process a dedicated memory pool or memory region. This segmentation reduces interference between processes, simplifies memory management, and prevents fragmentation while maintaining high processing throughput through parallel operation.
Solution Approach 2:
The patent adds a dimensional layer to memory management by introducing memory pools organized in hierarchical or dimensional structures. This allows multiple processes to operate in parallel with well-defined memory boundaries, simplifying memory management complexity while maintaining high throughput.
Data Source
AI summary
Embodiments store transaction metadata in dedicated pools of allocated memory chunks. Portions of the pools of allocated memory chunks are dedicated to the respective apply slave processes that mine and process change records. Also, the pools of allocated memory chunks are anchored within the structure of a transaction log such that buffering and application of metadata for one transaction does not block required buffering and application of metadata for other transactions. The standby database system pre-processes transaction metadata in preparation for application of the metadata to invalidate appropriate portions of MF data. Further, embodiments divide the work of pre-processing invalidation records among the many apply slave processes that record the invalidation records. A garbage collection selects memory chunks for garbage collection in reverse order of how the chunks were allocated. Also, a deduplication algorithm ensures that typically only a single invalidation message per block is applied to invalidate MF data.