Capture Apply Log Mining Direct Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In database systems that support information sharing through log mining, the publisher-subscriber model often leads to memory interference and pressure due to concurrent access to shared staging areas, resulting in inefficient memory usage and processing capacity.
Innovation Solution
Implementing a direct communication model between capture and apply processes, where the capture process mines and transfers log information to the apply process, eliminating the need for latches and allowing only one writer and one reader to access the memory structure, thereby reducing latency and improving processing capacity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple processes concurrently access the same staging area using a publisher-subscriber model, then information sharing capability is improved, but memory interference and memory pressure increase
Solution Approach 1:
The patent divides the single shared staging area into multiple separate staging areas, one for each capture process. This segmentation eliminates memory interference between processes while maintaining information sharing capability through the coordinated use of these separate areas by respective capture and apply processes.
Solution Approach 2:
The patent introduces coordination mechanisms including latches and sequence numbers that act as intermediaries to manage access to staging areas. These intermediaries enable controlled concurrent access without direct memory interference, allowing multiple processes to share information while preventing harmful conflicts.
2Reliability
If latches are used to coordinate concurrent accesses to the staging area, then access coordination is improved, but processing latency increases
Solution Approach 1:
The patent implements preliminary coordination through sequence numbers and predetermined access protocols. Processes are assigned specific staging areas and follow predetermined access patterns, eliminating the need for runtime latch acquisition and release operations that cause latency.
Solution Approach 2:
The patent uses sequence numbers as copies of position information to coordinate access without requiring processes to directly latch onto the same memory structures. Each process tracks its position through copied sequence information rather than through direct memory locking.
3Reliability
If all records in the same transaction are kept resident in shared memory, then transaction integrity is maintained, but memory space consumption increases
Solution Approach 1:
The patent segments transactions into individual records that can be processed and removed from memory independently. Each record is handled separately in its designated staging area, allowing memory consumption to be minimized while maintaining transaction integrity through proper sequencing and application of segmented records.
Solution Approach 2:
The patent enables discarding of processed records from shared memory once they have been successfully applied. Records are removed from memory after application, freeing up space for new records while ensuring transaction integrity through the sequence number mechanism that tracks what has been applied.
4Productivity
If a direct communication model is used between capture and apply processes, then processing capacity is improved, but system complexity increases
Solution Approach 1:
The patent creates a universal interface and protocol that works for both direct communication and publisher-subscriber models. The same staging area structures, sequence number mechanisms, and coordination protocols serve multiple communication patterns, reducing overall system complexity while enabling high-capacity direct communication.
Data Source
AI summary
In accordance with an embodiment of the present invention, an information sharing mechanism comprising a memory structure may be provided in a database system. In an embodiment, mined information transferred by a capture process to an associated apply process can be written into the memory structure without taking any latch. Similarly, the mined information can be read by the apply process from the memory structure without taking any latch. The capture and apply processes may work cooperatively to establish a safe point in log mining under various circumstances such as in an initial startup state, in a steady state, in a process restart scenario in the middle of information sharing. In some embodiments, the information sharing mechanism supports both checkpoint-free and checkpoint modes of log mining by the capture process. In addition, both the capture process and the apply process may employ an eager apply approach to increase processing capacity.