Persistent Memory Transaction Manager with Hash Directory Commit
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional database systems face bottlenecks due to the slow speed of conventional storage devices compared to volatile memory, leading to inefficiencies in data processing and handling system failures, and lack effective organization for minimizing unnecessary data writes and rewrites.
Innovation Solution
A system utilizing persistent memory devices like SCM or PCM for transaction management, which creates an in-memory log of transactions and commits a copy to a hash directory, allowing for efficient data processing, rollback capabilities, and minimizing interactions between volatile and persistent memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is written frequently to conventional disk storage for transaction persistency, then data durability is improved, but transaction processing speed deteriorates due to the slow speed of disk storage compared to volatile memory
Solution Approach 1:
The patent segments the storage system into two distinct layers: a fast volatile memory layer (DRAM) for active transaction processing and a slower persistent storage layer (disk or SCM) for durability. This segmentation allows each layer to operate independently at its optimal speed, with the volatile memory handling high-speed transactions and the persistent storage providing durability assurance, thereby resolving the speed-durability contradiction.
Solution Approach 2:
The patent introduces a write buffer as an intermediary component between the volatile memory and persistent storage. This buffer acts as a mediator that temporarily holds data during transaction processing, allowing transactions to be committed quickly to volatile memory while asynchronously flushing data to persistent storage in the background. This intermediary mechanism decouples the speed requirements of transaction processing from the durability requirements of persistent storage.
2Reliability
If traditional logging procedures are used to track transactions, then transaction reliability is improved, but processing speed deteriorates due to latency in committing transactions to disk memory
Solution Approach 1:
The patent implements preliminary action by pre-allocating buffer spaces and preparing log structures in advance within the volatile memory. Transaction logs are buffered and prepared for writing before actual commit operations occur, allowing the system to quickly acknowledge transaction commits to clients while performing the actual persistent storage operations asynchronously. This preliminary preparation eliminates wait times during commit operations.
Solution Approach 2:
The patent maintains continuity of useful action by implementing asynchronous background processes that continuously flush buffered transactions to persistent storage. Instead of blocking transaction processing to wait for disk writes, the system continuously performs useful work in the background (flushing logs and buffers) while maintaining forward progress on transaction processing, thereby eliminating idle time and reducing overall latency.
3Reliability
If conventional database management systems lack organization and space management, then data persistency is achieved, but memory cells break down over time causing data inaccuracies due to unnecessary writes and rewrites
Solution Approach 1:
The patent implements feedback mechanisms through version tracking and validation protocols that monitor the state of persistent storage. The system tracks versions of data records and validates data integrity after write operations, providing feedback loops that detect and correct potential data inaccuracies. This feedback system ensures that even if memory cells degrade over time, the system can identify and recover from data corruption, maintaining both persistency and accuracy.
Data Source
AI summary
Embodiments of a system are described. In one embodiment, the system is a device for performing operations and supporting transactions. The device is configured to receive a transaction comprising a command and data. The device writes the data to a transaction manager on a persistent memory device. The transaction manager also maintains a status of the transaction and reference to entries within memory that are manipulated by the transaction. The device also creates an in-memory log of the transaction in a first hash directory. The device then commits a copy of the first hash directory to a second hash directory maintained on a persistent memory device.