Multi-Cluster Database Deployment with Cross-Cluster Data Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional storage systems face inefficiencies in data management and reliability due to redundant write operations and lack of direct control over storage processes, particularly in flash-based systems, leading to increased latency and reduced reliability.
Innovation Solution
Implementing a direct-mapped flash storage system where higher-level processes, such as the operating system, manage data blocks directly without address translation by storage controllers, and utilizing non-volatile RAM as a buffer to improve write latency and reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If storage controllers perform address translation for flash drives, then data management is simplified for higher-level processes, but write latency increases and write operations become redundant
Solution Approach 1:
The patent extracts the address translation function from the storage controller and implements it directly in the host system's memory management unit (MMU). This removes the unnecessary intermediary translation layer in the storage controller, allowing direct mapping between host virtual addresses and flash drive physical addresses, thereby eliminating redundant write operations and reducing latency while maintaining simplified data management at the host level.
Solution Approach 2:
The patent introduces a new intermediary mechanism - a host-based translation layer with a page table structure that maps virtual addresses to physical addresses directly in the host system. This intermediary eliminates the need for storage controller-based translation by handling address mapping at the host level, reducing the number of translation steps and eliminating redundant writes to the storage controller's translation layer.
2Ease of operation
If storage controllers manage data blocks with address translation, then storage access is abstracted, but reliability decreases due to redundant operations
Solution Approach 1:
The patent extracts the address translation management from the storage controller and relocates it to the host system's MMU. This extraction eliminates the reliability issues caused by redundant translation operations in the storage controller while maintaining storage access abstraction through the host-based page table mechanism. The single-source translation management improves consistency and reduces errors.
Solution Approach 2:
The host system's MMU performs self-service address translation using its own page table structures, eliminating dependency on the storage controller's translation layer. This self-service approach ensures that address mapping is performed once in the host system, improving reliability by avoiding the redundancy and potential inconsistencies that arise when multiple components perform translation.
3Reliability
If multiple write operations are performed through storage controllers, then data is ensured to be written, but energy consumption increases
Solution Approach 1:
The patent extracts the write operation management from the storage controller and implements direct host-to-flash writing through the host's MMU. This extraction eliminates redundant write cycles through the storage controller by establishing a direct write path from host memory to flash storage, ensuring data is written reliably while significantly reducing the energy consumption associated with multiple controller-mediated write operations.
Data Source
AI summary
Systems, apparatuses, and methods of multi-cluster database deployment are disclosed. An illustrative method includes receiving, by a controller, a request to deploy a database to multiple clusters; determining, by the controller and based on the request, a set of clusters to which to deploy the database, wherein the determining is based on the set of clusters including a first cluster and a second cluster having a relationship established using a storage system implemented on the first cluster and the second cluster; and deploying, by the controller, the database to the set of clusters, wherein the deploying the database to the set of clusters comprises directing the storage system implemented on the first cluster and the second cluster to replicate data of the database between the first cluster and the second cluster.