Database Middleware Shard Processing for Distributed Query Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data management systems face increasing complexity when dealing with distributed databases, requiring frequent changes to applications as tables are divided or databases are expanded, leading to inefficiencies in querying and data storage.
Innovation Solution
A data management system utilizing database middleware with a shard processing module that creates a shard key identification and selects appropriate databases using shard metadata, reducing application complexity by distributing queries across multiple databases and maintaining connections through modules like job queues and connection pooling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If tables are divided and data is diffusively stored throughout several databases, then data storage capacity and query flexibility are improved, but application complexity increases
Solution Approach 1:
The patent introduces database middleware as an intermediary layer between the application and multiple databases. The middleware automatically handles table division, data distribution, and query routing across databases using metadata and shard key IDs. This mediator absorbs the complexity of distributed data management, allowing the application to interact with a unified interface while the middleware manages the underlying complexity of data diffusion across multiple databases.
2Adaptability or versatility
If databases are expanded or tables are additionally divided, then data storage and query capabilities are improved, but application changes are required
Solution Approach 1:
The patent implements preliminary action by pre-defining metadata that describes the logical structure of tables and their distribution across databases. The middleware uses this pre-established metadata to automatically route queries and manage data distribution. When databases are expanded or tables divided, only the metadata needs to be updated, not the application itself. The middleware automatically adapts to structural changes using the updated metadata, eliminating the need for application modifications.
3Adaptability or versatility
If direct database searching is implemented in applications, then database selection flexibility is improved, but application complexity and maintenance burden increase
Solution Approach 1:
The patent extracts the database selection and routing logic from the application and places it in the middleware layer. The middleware creates shard key IDs and uses metadata to determine which database should handle each query, completely removing this complexity from the application. The application simply issues queries to the middleware, which automatically extracts and processes the routing information, selecting the appropriate database without application involvement.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided is a data management system using database middleware (230), wherein the database middleware may include a shard processing module (231) to create a shard key identification (ID) based on a query input, and select a single database of a plurality of databases (220) through a shard ID determined using the created shard key ID and shard metadata.