Remote Data Transfer via Memory Access Protocol
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data transfer systems experience bottlenecks due to high latency and overhead when transferring data from a server to a client, particularly when using network protocols like HTTP and TCP, which impede the client's ability to process data efficiently.
Innovation Solution
Implementing a data transfer scheme that uses a memory access protocol, such as RDMA, to directly transfer data between network targets and the client's memory, bypassing the network stack and reducing latency, while also employing error correction techniques like erasure coding to ensure data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is transferred using network protocols like HTTP and TCP through the server, then data transfer can be performed with standard networking infrastructure, but latency and overhead increase, impeding client data processing efficiency
Solution Approach 1:
The patent extracts the data transfer path from the traditional server-mediated route and creates a direct peer-to-peer connection between client and target. This removes the server from the data path, eliminating network stack processing and reducing latency while maintaining reliability through protocol design
Solution Approach 2:
The patent introduces a custom protocol that acts as an intermediary mechanism enabling direct communication between client and target. This protocol establishes connection rules and data transfer semantics without requiring traditional server mediation, thus reducing overhead while preserving transfer reliability
2Adaptability or versatility
If data is transferred through the server using network stacks, then standard networking can be used, but overhead increases and processing efficiency decreases
Solution Approach 1:
The patent extracts the essential connectivity function from complex network stacks and implements a simplified direct connection mechanism. This removes unnecessary protocol layers and processing overhead while maintaining the core ability to establish and maintain data transfer paths
Solution Approach 2:
Instead of following the traditional top-down approach where applications use standard network stacks, the patent inverts the model by implementing a specialized low-level protocol directly at the transport layer, bypassing upper-layer protocol overhead and simplifying the communication path
3Loss of time
If direct memory access protocol is used for data transfer, then latency is reduced, but additional mechanisms are needed for error correction and data integrity
Solution Approach 1:
The patent applies preliminary error correction encoding to data before transmission. By pre-processing data with erasure coding or similar techniques, the system prepares redundant information in advance, enabling error recovery without adding transmission latency while maintaining data integrity
Solution Approach 2:
The patent implements beforehand cushioning by incorporating error correction codes and redundancy mechanisms prior to data transfer. This preemptive approach creates a buffer against potential transmission errors, allowing recovery without retransmission and thus not increasing latency
Data Source
AI summary
A method may include receiving, at a target, from a server, a command, information to identify data, and access information to perform a data transfer using a memory access protocol, and performing, based on the command, based on the access information, the data transfer between the target and a client using the memory access protocol. The information to identify the data may include an object key, and the object key and the access information may be encoded, at least partially, in an encoded object key. The method may further include sending, based on the data transfer, from the target to the server, a completion. The method may further include sending, based on the completion, from the server to the client, an indication of success. The method may further include reconstructing the data based on the parity data.