RDMA Hardware Compression with Method Tags for Data Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current RDMA technologies face challenges in maximizing network bandwidth utilization and transmission efficiency due to the high volume of data transmitted over networks, leading to suboptimal performance in large-scale data centers.
Innovation Solution
Implementing a method where RDMA devices compress raw data using a preset compression method, encapsulate the compressed data into data packets with a method tag, and transmit these packets, allowing the receiving end to decompress the data using the corresponding algorithm, thereby reducing the data volume and improving transmission efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transmitted without compression in RDMA networks, then data transmission simplicity is maintained, but network bandwidth utilization and transmission efficiency deteriorate due to high data volume
Solution Approach 1:
The patent applies preliminary action by performing data compression before transmission. The sending-end RDMA device compresses the data packet using a compression algorithm before sending it over the network, reducing the data volume that needs to be transmitted. This preliminary compression action directly addresses the contradiction by minimizing the quantity of substance (data volume) while maintaining or improving productivity (transmission efficiency).
Solution Approach 2:
The patent applies parameter changes by transforming the data from an uncompressed state to a compressed state, changing the physical parameter of data volume. The compression algorithm modifies the representation of the data, reducing its size while preserving the essential information, thereby resolving the contradiction between data volume and transmission efficiency.
2Quantity of substance
If data compression is implemented at the sending end, then data volume for transmission is reduced, but device complexity increases due to additional compression and decompression operations
Solution Approach 1:
The patent applies self-service by enabling the RDMA devices to perform compression and decompression operations autonomously. The sending-end device automatically compresses data before transmission, and the receiving-end device automatically decompresses incoming compressed data. This self-service mechanism reduces the need for external intervention or complex coordination, thereby limiting the increase in device complexity while achieving data volume reduction.
3Reliability
If compressed data is transmitted with method tags, then data integrity is ensured through accurate decompression, but packet structure complexity increases
Solution Approach 1:
The patent introduces method tags as an intermediary element that facilitates reliable decompression without significantly complicating the overall system. The method tag acts as a mediator that carries compression algorithm identification information, enabling the receiving-end device to select the appropriate decompression algorithm. This intermediary approach ensures data integrity while keeping the packet structure relatively simple and manageable.
Data Source
AI summary
A Remote Direct Memory Access (RDMA) data sending method is disclosed. The method is applicable to a sending end, with a data-transmission RDMA device disposed thereon. The method includes: the data-transmission RDMA device acquiring raw data; the data-transmission RDMA device compressing the raw data by using a preset compression method to obtain compressed data; and the data-transmission RDMA device encapsulating the compressed data into a data packet, and transmitting the data packet to a receiving end. The data packet may include a method tag corresponding to the preset compression method. In this method, the compression and transmission of the raw data are conducted by the data-transmission RDMA device on the hardware level.