Networking Switch Object Storage Intelligence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current object storage systems face a tradeoff between reliability and performance due to the overhead of replication, which increases internal network traffic and slows down data access times as the number of replicas grows, particularly in large mission-critical environments.
Innovation Solution
Embedding object ID generation and storage node intelligence within networking switches, such as TOR and EOR units, to consolidate and route data copies more efficiently, reducing end-to-end traffic flow and internal network traffic.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the number of replicas is increased to improve reliability, then storage system reliability is improved, but internal network traffic increases and data access speed decreases
Solution Approach 1:
The patent introduces an intermediary hashing function that maps object IDs to storage node identifiers, enabling direct routing of data to the correct storage node without requiring full end-to-end routing flows. This intermediary mechanism reduces network traffic overhead while maintaining reliable access to replicated data across multiple storage nodes.
Solution Approach 2:
The patent segments the routing function into two parts: the hashing function that determines storage node identification, and the routing function that directs data flow. This segmentation allows the system to handle multiple replicas efficiently by computing storage node identifiers independently for each replica, reducing the overhead of end-to-end routing while maintaining reliability.
2Reliability
If the number of replicas is increased to improve reliability, then storage system reliability is improved, but internal network traffic overhead increases
Solution Approach 1:
The hashing function serves as an intermediary that computes storage node identifiers locally without requiring extensive network communication. By mapping object IDs to storage node identifiers through this intermediary function, the system reduces the overhead of coordinating multiple replicas across the network while maintaining reliability through proper replication.
Solution Approach 2:
The system performs preliminary computation of storage node identifiers using the hashing function before actual data transmission. This preliminary action determines the destination storage nodes for each replica in advance, reducing the overhead of dynamic routing decisions and network traffic coordination during data operations.
3Reliability
If end-to-end routing flows are used for each data copy, then data can be reliably routed to storage nodes, but user access times increase
Solution Approach 1:
The routing function acts as an intermediary that uses pre-computed storage node identifiers to direct data flow efficiently. Instead of implementing full end-to-end routing flows that require extensive communication and coordination, the routing function simply directs data to the storage node identified by the hashing function, significantly reducing access time while maintaining routing reliability.
Solution Approach 2:
The system copies only the necessary routing information (storage node identifier) from the hashing function result to guide data transmission, rather than copying and processing complete end-to-end routing paths. This selective copying of essential routing information reduces the time required for data access while ensuring reliable delivery to the correct storage node.
Data Source
AI summary
A method performed by a networking switch in an object storage system. The method includes receiving a first packet from a network comprising an object ID and a data object. The method includes generating a replica for the data object. The method includes generating an object ID for the replica of the data object. The method includes determining a destination storage node for the replica of the data object. The method includes sending a second packet from the networking switch to the destination storage node. The second packet includes the object ID for the replica of the data object and the replica of the data object.