SSD Redo Log Processing for Low-Latency Database Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The traditional separation of compute and storage in database systems, particularly with SSDs, leads to high communication bandwidth usage and latency due to the need for hosts to manage data and redo logs, which is inefficient and undesirable.
Innovation Solution
Offload redo log processing from hosts to SSDs, where the SSDs update and create data pages based on redo log entries, reducing the need for data pages and undo logs to be written from the host, and enabling parallel processing of redo entries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If hosts manage data and redo logs in traditional database systems, then data integrity is maintained, but communication bandwidth usage increases and latency increases
Solution Approach 1:
The patent extracts redo log processing functionality from the host system and relocates it to the SSD. The SSD now independently manages redo log buffering, parsing, and application to data pages, removing this computational burden from the host and reducing the need for continuous data transfer between host and storage, thereby conserving communication bandwidth while maintaining data integrity through the SSD's autonomous processing capabilities
2Productivity
If hosts manage redo logs in traditional database systems, then redo log processing is performed, but latency increases due to data transfer requirements
Solution Approach 1:
The SSD is equipped with dedicated redo log processors that enable it to autonomously buffer, parse, and apply redo logs without requiring continuous host intervention. The SSD independently manages the redo log buffer, processes redo entries, and updates data pages in place, eliminating the latency associated with repeated host-storage communication cycles while maintaining efficient redo log processing throughput
3Quantity of substance
If data pages and undo logs are written from host to storage, then data is stored, but communication bandwidth is consumed
Solution Approach 1:
The patent extracts the function of data page creation and update operations from the host and transfers it to the SSD. The SSD's redo log processors directly create and update data pages within the storage medium using redo log information, eliminating the need for hosts to assemble and transfer complete data pages to storage, thereby reducing communication bandwidth consumption while maintaining full data storage capability
Solution Approach 2:
The redo log buffer acting as an intermediary structure within the SSD enables efficient data page reconstruction. Instead of transferring complete data pages from host to storage, the SSD uses the redo log buffer as an intermediate workspace to reconstruct and update data pages in place, significantly reducing the volume of data that needs to be transferred over the communication interface while ensuring data is properly stored and maintained
Data Source
AI summary
Methods, systems, and computer-readable storage media for receiving, by a SSD from a host, a write command to write content to memory, the content including a redo log of a database system, the redo log including a set of redo entries, each redo entry recording one or more changes to data in the database system, writing, by a redo log processor executed by the SSD, redo entries of the redo log to a redo log buffer, the redo log buffer comprising a page identifier (page ID) ordered array and a set of bidirectional linked tables, the redo entries being written to the bidirectional linked tables in the set of bidirectional linked tables, and executing, by the redo log processor, one or more of creating a data page and updating a data page in memory of the SSD, the data page being stored in memory of the SSD.