Clustering Layer for Database Sharding in Multi-Node Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Horizontally scaling databases in multi-node clusters is complex and often requires significant modifications to the database structure and existing application layers, which can be costly and inefficient, especially in agile development environments where downtime is a concern.
Innovation Solution
A clustering layer is introduced between the API layer and the service layer to intercept API calls, determine the need for database sharding based on changes in the number of nodes, and automatically shard the database without modifying the database structure or existing application layers, allowing for horizontal scaling and load balancing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If horizontal scaling is implemented by modifying database structure and application layers, then database capacity and performance are improved, but system complexity and implementation cost increase significantly
Solution Approach 1:
The patent introduces a clustering layer as an intermediary component between the API layer and service layer. This clustering layer handles all database sharding and node discovery logic, allowing the existing application layers to remain unchanged while enabling horizontal scaling. The clustering layer acts as a mediator that translates API calls into appropriate database operations across multiple nodes, thus improving database capacity without increasing system complexity.
2Adaptability or versatility
If database sharding is implemented with structural modifications, then horizontal scaling capability is improved, but implementation time and potential downtime increase
Solution Approach 1:
The patent implements automatic node discovery and dynamic sharding through the clustering layer, which proactively monitors cluster composition and automatically redistributes data when nodes are added or removed. This preliminary and continuous action eliminates the need for manual intervention and reduces implementation time, as the system adaptively scales horizontally without requiring downtime for structural modifications.
3Productivity
If existing application layers are modified to support sharding, then database distribution across nodes is improved, but development cost and risk increase
Solution Approach 1:
The patent segments the system into distinct layers: the clustering layer handles all sharding and node management functionality, while the existing API and service layers remain intact. This segmentation allows database distribution across nodes to be improved through the specialized clustering layer without requiring modifications to existing application layers, thereby reducing development cost and risk.
Data Source
AI summary
Examples include a multi-node cluster having a node with a clustering layer. The clustering layer may be located between an application programming interface (API) layer and a service layer and the multi-node cluster may be associated with a database. In some examples, the clustering layer may discover whether a number of nodes associated with the multi-node cluster has changed. Based, at least in part, on the determination that the number of nodes associated with the multi-node cluster has changed, at the clustering layer, the database may be sharded and a new API call may be issued to the API layer.