Real Time Database Subpartitioning for Fault Resilience
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current real-time database systems require excessive resources and infrastructure costs due to the need for multiple redundant replicas to ensure high availability and scalability, leading to inefficiencies and sub-optimal capacity utilization, especially during failures and recovery processes.
Innovation Solution
A real-time database system that partitions data into multiple master databases and further divides each partition into subpartitions, with replicas stored on different physical storage units, allowing for efficient distribution of traffic and replication, reducing the need for idle capacity and enabling N+k redundancy with minimal extra capacity for fault resilience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple redundant replicas are provided to ensure high availability and scalability, then system reliability is improved, but infrastructure costs and resource consumption increase linearly
Solution Approach 1:
The patent segments the database system into master databases and replica databases, where each master database manages specific data partitions and each replica database serves as backup for specific masters. This segmentation allows precise control over redundancy levels, enabling the system to maintain high availability while optimizing resource allocation by not requiring all replicas to be fully dimensioned for all failure scenarios simultaneously.
Solution Approach 2:
The patent implements dynamic role assignment where replica databases can become master databases when failures occur. The system dynamically adjusts the number of active masters and replicas based on current system state and failure conditions, allowing infrastructure resources to be utilized more efficiently rather than maintaining static over-provisioning for worst-case scenarios.
2Reliability
If multiple redundant replicas are provided to withstand simultaneous failures, then system reliability is improved, but device complexity increases
Solution Approach 1:
The patent makes replica databases multi-functional: they serve as backups for their designated master databases during normal operation, and can simultaneously serve as masters for their own data partitions when needed. This multi-functionality reduces the total number of dedicated components needed while maintaining resilience against multiple simultaneous failures.
3Reliability
If replicas are dimensioned for extra tasks during recovery, then system reliability is improved, but productivity decreases due to idle capacity
Solution Approach 1:
The patent ensures that replica databases continuously perform useful work by serving read requests and maintaining their own data partitions, even while providing backup capacity. During recovery operations, the system continuously distributes traffic across available masters and replicas, ensuring no capacity sits idle while maintaining recovery capability.
4Reliability
If the system is dimensioned for big traffic in faulty state, then reliability is improved, but infrastructure costs increase due to extra capacity requirements
Solution Approach 1:
The patent changes the operational parameters of the system dynamically based on failure state. In normal operation, the system operates with optimal performance parameters. When failures occur, the system reconfigures traffic distribution and replica promotion to maintain service levels, avoiding the need to permanently dimension the system for worst-case scenarios.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention relates to a real time database system configured to store a database content with a plurality of data sets, the database content being partitioned in different partitions (P1-P5). The system comprises a plurality of master databases (10-50), each master database containing a partition of the database content, each partition being provided on a different physical storage unit and at least one replica database (110-510), each replica database containing subpartitions (11-54) of the partitions stored in master databases (10-50). Furthermore, the system comprises at least one distributor (1-5) configured to route a request for a data set to the master database (10-50), where the data set to which the request refers to, is provided, and distributor (1-5) configured to divide each partition (P1-P5) into several subpartitions. Additionally, the system comprises at least one replicator (100-500), configured to generate a replica (11-54) of each subpartition, and configured to store each replica (11 -54) on a replica database (110-510), wherein the at least one replicator (100-500) stores the replicas (11 -54) of each subpartition in such a way that the replicas of one partition are contained on different physical storage units, on a physical storage unit other than the corresponding partition (P1-P5), and on a physical storage unit other than the replicator (100-500) generating the replica (11-54) of said one partition, wherein the distributor is configured to route a replication request for a subpartition to the replicator responsible for said subpartition.