Zero-Copy Object Serialization via DMA and RDMA
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Serialization and deserialization processes in digital information transmission across data networks are resource-intensive, leading to high memory consumption and increased processor usage due to data copying, which results in inefficiencies.
Innovation Solution
The use of Direct Memory Access (DMA) and Remote Direct Memory Access (RDMA) techniques to transmit metadata and addresses of data members, allowing for the creation of a new object instance without the need for duplicate memory allocation and data copying, thereby reducing memory copy overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional serialization and deserialization processes are used to transfer objects across a data network, then objects can be converted into byte sequences and reconstructed, but memory consumption increases due to duplicate storage and processor usage increases due to additional copying operations
Solution Approach 1:
The patent extracts only the essential metadata and addresses of data members from the original object, transmitting these through the network while leaving the actual data in the original memory location. This eliminates the need to create complete duplicate copies of the object in memory during serialization and deserialization, thereby reducing memory consumption while preserving the ability to reconstruct the object.
Solution Approach 2:
The patent introduces an intermediary mechanism where metadata and addresses serve as proxies for the actual data. Instead of copying the entire object, the system transmits these intermediary representations that can be used to locate and reconstruct the original data, reducing the quantity of data that needs to be stored and transmitted.
2Reliability
If traditional serialization and deserialization processes are used to transfer objects across a data network, then objects can be converted into byte sequences and reconstructed, but processor usage increases due to additional copying operations
Solution Approach 1:
The patent extracts only the essential metadata and addresses from the object, avoiding the need to perform extensive copying operations on the actual data. This extraction approach significantly reduces the number of processor operations required during serialization and deserialization, thereby reducing energy consumption while maintaining the ability to accurately reconstruct objects.
Solution Approach 2:
The patent performs preliminary actions by pre-calculating and storing metadata and addresses during object creation or before transmission. This preliminary preparation eliminates the need for time-consuming data copying operations during the actual serialization and deserialization processes, reducing processor usage and energy consumption.
3Ease of operation
If data is copied during serialization and deserialization, then objects can be reconstructed in memory, but memory copy overhead increases resulting in high memory consumption and cache pollution
Solution Approach 1:
The patent extracts only the necessary metadata and address information from the object, avoiding the need to copy the entire data set. This extracted information is sufficient to locate and reconstruct the original object in memory, thereby maintaining ease of operation while significantly reducing memory copy overhead and preventing cache pollution.
Solution Approach 2:
The patent uses a form of selective copying where only metadata and addresses are copied to the byte sequence, rather than copying the actual data. This minimal copying approach maintains the ability to reconstruct objects while dramatically reducing the memory overhead associated with traditional full-data copying methods.
Data Source
AI summary
Serialization and deserialization of an object are performed by transmitting metadata and addresses of data members in a byte stream through a data network, receiving the byte stream from the data network, defining a container for the object, obtaining the addresses of the data members in the first memory from the input byte stream, applying direct memory access (DMA) or remote direct memory access (RDMA) to read the data members using the obtained addresses, and writing the data members into the container to create a new instance of the object.