Clustered Storage Routing with Precomputed Weights for Lower Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Legacy distributed data storage clustering solutions face inefficiencies in data placement and request routing, leading to increased latencies and suboptimal performance in retrieving data records from clustered storage servers.
Innovation Solution
A computer-implemented method that determines the optimal storage cluster for data retrieval based on weights associated with each cluster, utilizing a multi-tier storage architecture with high-performance local storage for frequently accessed data and cluster mapping to enhance retrieval efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is distributed across multiple storage clusters, then scalability and redundancy are improved, but data retrieval latency increases
Solution Approach 1:
The system pre-computes and stores optimal cluster selection weights in a storage medium before data retrieval requests arrive. When a retrieval request is received, the system queries the pre-stored weights to quickly determine the optimal cluster, avoiding real-time computation delays and reducing data retrieval latency while maintaining redundancy across multiple clusters
Solution Approach 2:
The patent introduces an intermediary storage medium that holds pre-computed cluster weights and mapping information between the query interface and storage clusters. This intermediary layer enables fast lookup of optimal clusters without directly querying multiple storage clusters for each data retrieval request, thus reducing latency while preserving redundancy
2Reliability
If data is distributed across multiple storage clusters, then scalability is improved, but request routing complexity increases
Solution Approach 1:
The system transforms the complex multi-cluster routing problem into a simpler parameter-based selection process. By pre-computing weight parameters for each cluster based on various factors (load, performance metrics, geographic distribution) and storing them in an intermediary storage medium, the routing complexity is reduced to a simple weight comparison and lookup operation, enabling scalable cluster addition without proportionally increasing routing complexity
Solution Approach 2:
The system performs preliminary computation of cluster weights and stores mapping information in advance. This pre-computation includes analyzing cluster performance metrics, load balancing requirements, and data placement information to determine optimal weights before retrieval requests arrive. When requests come in, the system only needs to query the pre-stored weights and select the optimal cluster, significantly simplifying the routing logic while supporting scalable cluster configurations
3Reliability
If traditional distributed storage clustering is used, then data redundancy is achieved, but data placement efficiency decreases
Solution Approach 1:
The system implements feedback mechanisms where cluster performance metrics, load information, and data access patterns are continuously monitored and used to update cluster weights in the intermediary storage medium. This feedback loop enables dynamic optimization of data placement decisions, improving placement efficiency while maintaining redundancy across clusters. The weights are recalculated and updated based on actual system performance, allowing the system to adapt to changing conditions and optimize data placement automatically
Data Source
AI summary
This disclosure describes techniques for enabling efficient and/or expeditious retrieval of data records stored across a clustered storage framework. An example system manages storage of data records in the storage clusters based on predefined placement rules. Frequently accessed data records are stored in a local high-performance store storage medium to reduce retrieval latency. The system determines which storage cluster to retrieve a requested data record from by selecting the cluster with the highest weight, as determined based on performance-related metrics like cluster utilization and geographic proximity.