Federated Sharding Layer for Database Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Migrating from multiple siloed databases to a traditional sharded database is disruptive and violates business requirements, as it requires unification of database schema, loss of management autonomy, and massive data migration.
Innovation Solution
Integration of existing component databases into a federated sharded database without disruption, achieved by creating a shard catalog that defines each database as a shard, mirroring local schemas to create a global schema, and using pseudocolumns for query routing, allowing seamless data access across regions without data migration or schema changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional sharding migration is performed, then centralized data management is achieved, but data migration disruption and loss of management autonomy occur
Solution Approach 1:
The patent introduces a federated sharding layer as an intermediary between existing databases and the application layer. This federated layer provides centralized management capabilities through a unified query interface while the underlying databases maintain their independence and continue operating without migration disruption.
Solution Approach 2:
The system segments the database architecture into independent component databases that retain their autonomy, while introducing a federated sharding layer that provides centralized management. This segmentation allows each database to maintain its own schema and management policies while being accessible through a unified interface.
2Ease of operation
If database schema unification is performed, then centralized administration is enabled, but management autonomy of individual databases is lost
Solution Approach 1:
The patent applies local quality by allowing each component database to maintain its own schema and management characteristics (local autonomy) while the federated sharding layer provides centralized administration capabilities. Each database can have different schemas, and the federated layer adapts to these local variations through dynamic schema generation.
Solution Approach 2:
The federated sharding layer serves multiple functions: it provides centralized administration through unified query interfaces, maintains compatibility with diverse underlying database schemas, and enables cross-database operations. This multi-functionality allows centralized management without imposing a single schema on all component databases.
3Adaptability or versatility
If massive data migration is performed, then single logical database is created, but business requirements violation and disruption occur
Solution Approach 1:
The patent implements preliminary action by establishing the federated sharding layer before any data migration would be required. This layer immediately provides single logical database functionality through unified query interfaces, allowing applications to access data across multiple databases without actual data movement, thus maintaining business continuity.
Solution Approach 2:
The system creates a virtual copy of the data access layer through the federated sharding interface. Instead of physically moving data between databases, the federated layer creates virtual representations that allow unified access to data distributed across multiple independent databases, eliminating the need for disruptive data migration.
Data Source
AI summary
Techniques described herein create a sharding environment from existing independent databases, use the sharding environment to issue cross shard queries, create “on all shards” new objects, and create and “on all shards” new users that will have permission to access the data across all database shards.