MultiDimensional Data Sharding in Distributed Databases
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current multi-host networked database systems face performance limitations in timely data storage, access, and processing, especially during peak hours, due to inconsistent computational throughput, which increases with growth, and existing distributed databases do not effectively isolate different types of data processing, leading to costly scaling solutions that do not guarantee uninterrupted real-time data processing.
Innovation Solution
The method involves dividing customer data into distinct data type portions and routing them to designated groups of servers based on access requirements, using a logical routing layer to ensure each data type is stored on separate servers optimized for its specific needs, such as high-access index data and low-access historical data, with appropriate hardware configurations and replication factors for redundancy and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all data categories are stored together in each server, then data redundancy is simplified, but processing performance deteriorates due to resource contention between different data types
Solution Approach 1:
The patent segments data storage by separating different data categories (index data, real-time data, historical data, sensitive data) across different servers. Each server is dedicated to storing specific data types, which eliminates resource contention and improves processing performance while maintaining redundancy through strategic replication of critical data across multiple servers.
Solution Approach 2:
The patent applies local quality by assigning different hardware configurations and server resources to different data types based on their specific access patterns and processing requirements. Real-time data receives higher resource allocation for low-latency access, while historical data uses more economical resources for bulk processing, optimizing overall system performance.
2Productivity
If more servers are added to handle peak loads, then processing capacity increases, but system complexity and cost increase
Solution Approach 1:
The patent segments the database system into specialized servers for different data types, allowing each server to be optimized for its specific workload. This segmentation enables selective scaling - only the servers handling peak-load data types need additional capacity, rather than scaling the entire system, thereby reducing overall complexity and cost.
Solution Approach 2:
The patent implements dynamic resource allocation where servers can be selectively activated or scaled based on real-time demand patterns. The system dynamically routes requests to appropriate servers based on data type and current load, allowing flexible capacity management without requiring over-provisioned homogeneous server infrastructure.
3Adaptability or versatility
If data is distributed evenly across servers, then hardware requirements are standardized, but processing latency increases due to data access patterns
Solution Approach 1:
The patent applies local quality by assigning different hardware configurations to different servers based on the specific needs of each data type. Servers storing real-time data use high-performance storage and processing resources for low-latency access, while servers for historical data use more economical hardware. This non-uniform distribution optimizes access speed for each data type while maintaining overall system versatility.
Data Source
AI summary
Systems and methods are provided for storing customer data in a distributed database. The method including: dividing a customer's data into a plurality of different data type portions; routing the plurality of different data type portions to at least two different servers, wherein at least one of the plurality of different data type portions are routed to one of the at least two different servers and at least another of the plurality of different data portions are routed to another of the at least two different servers; and storing the plurality of different data type portions at the server to which they are routed, wherein each data type is associated with at least one of the at least two different servers based at least in part on access requirements of the customer's data for each data type.