Connection Relocation for Multi-Rack Storage Traffic Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed object storage systems face network congestion due to the shuffling of object data between racks during reads and writes, which can lead to performance degradation as the system grows, especially when data is not localized within the same rack.
Innovation Solution
Implementing a connection relocation mechanism within a multi-rack distributed storage system using a connection-oriented protocol like QUIC, which allows for the migration of client requests to the rack where the object data is located, reducing inter-rack data transfers and congestion by relocating the connection endpoint.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If object data is distributed across multiple racks in a distributed storage system, then storage capacity and fault tolerance are improved, but network congestion and performance degradation occur due to data shuffling between racks during reads and writes
Solution Approach 1:
The patent applies local quality by routing client requests directly to the rack where the requested object data is stored, rather than funneling all requests through a single rack. This creates different data flow paths for different racks, with each rack handling its own data locally. The connection-oriented protocol enables this localized data access pattern, reducing inter-rack network traffic and congestion while maintaining the distributed storage architecture's capacity and fault tolerance benefits
2Adaptability or versatility
If data is stored in a distributed manner across multiple racks, then system scalability is improved, but inter-rack data transfer increases causing congestion
Solution Approach 1:
The system performs preliminary action by maintaining metadata that tracks the location of object data across racks before client requests arrive. When a client requests data, the system can immediately determine which rack holds the data and route the request directly there, avoiding unnecessary data movement across the network. This preliminary organization of data location information enables efficient scalable growth without proportionally increasing inter-rack traffic
3Reliability
If object data is accessed from remote racks, then data availability is improved, but access time increases due to network traversal
Solution Approach 1:
The patent inverts the traditional client-server data access model by allowing the data to come directly to the client from the storage rack, rather than requiring the client to access data through an intermediate server rack. This inversion eliminates unnecessary network hops and reduces access time while maintaining data availability across the distributed system. The connection-oriented protocol enables this direct data path by maintaining stateful connections that can be routed optimally
Data Source
AI summary
An example system may include a first rack including a first access node and a second rack including a second access node and a storage node. The first and second racks are included within a multi-rack distributed storage system. The first access node is configured to receive over a connection with a client a first request for an object data. The connection with the client is uniquely identified by a connection ID independent of source and destination addresses and ports. The first access node is further configured to determine object data locality details of the object data and relocate the connection to the second access node in the second rack when the object data locality details of the object data indicate the object data is located within the second rack.