RDMA Data Transfer System Bypassing Pre-Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data transport systems require pre-processing of data, such as compression or manipulation, which leads to delays, data integrity issues, and corruption, especially when transferring large data sets across remote locations.
Innovation Solution
A high-efficiency, high-bandwidth data storage and transfer system that allows data to be moved 'as is' without pre-processing, utilizing a network of distributed points of presence with proprietary protocols like RDMA and InfiniBand over Converged Ethernet to eliminate latency and inefficiencies, ensuring secure and fast file transfers across multiple remote locations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is compressed or manipulated before transfer, then data size is reduced, but data integrity is compromised and transfer time increases
Solution Approach 1:
The patent extracts the data processing step from the transfer process by implementing RDMA technology that allows data to be moved directly between storage and compute resources without being processed, compressed, or manipulated in between, thereby preserving data integrity while enabling efficient transfer
Solution Approach 2:
The patent introduces an intermediary storage system with distributed points of presence that acts as a mediator between data sources and destinations, enabling direct memory access across network boundaries without requiring data manipulation at intermediate nodes
2Adaptability or versatility
If data is pre-processed before transfer, then adaptability to different formats is improved, but transfer time is delayed
Solution Approach 1:
The patent implements preliminary actions by pre-establishing network paths, configuring RDMA connections, and setting up distributed storage nodes before data transfer begins, so that no processing time is consumed during the actual transfer operation
Solution Approach 2:
The patent replaces mechanical data processing operations with direct memory access operations using RDMA and InfiniBand protocols, eliminating the need for software-based compression, manipulation, or format conversion during transfer
3Adaptability or versatility
If conventional TCP/IP protocols are used, then network compatibility is maintained, but transfer speed is limited
Solution Approach 1:
The patent implements a dynamic protocol stack that operates at multiple levels: RDMA at the transport layer for high-speed direct memory access, InfiniBand for low-latency communication, and Converged Ethernet for network transport, allowing the system to dynamically select optimal paths and protocols based on network conditions
Solution Approach 2:
The patent changes key network parameters by implementing custom RDMA protocols and InfiniBand configurations that bypass standard TCP/IP overhead, reducing latency and increasing throughput while maintaining compatibility through Converged Ethernet infrastructure
4Adaptability or versatility
If data is transferred across remote locations, then geographic coverage is expanded, but transfer time increases due to distance
Solution Approach 1:
The patent adds a new dimension to data transfer by implementing distributed points of presence across multiple geographic locations, creating a multi-dimensional transfer path that reduces physical distance and latency through strategic node placement and direct routing
Data Source
AI summary
Systems and apparatuses for performing fast file transfers and methods for making and using the same. In various embodiments, the system advantageously can eliminate distance constraints between multi-site computational environments, provide a dramatic reduction in transfer and/or dynamically provision bandwidth on a fabric of the system among other things.