Data Transfer Accelerator Using Operation Log Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data transfer between nodes results in performance loss and increased energy consumption, particularly as data transfer frequency and size increase.
Innovation Solution
Implementing data transfer accelerators that monitor memory operations, log and optimize operations by removing inverse and idempotent operations, and determine whether to send data or operation messages based on heuristics and memory address registrations, thereby reducing unnecessary data transfer and power usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is transferred frequently and in large sizes between nodes, then data availability and completeness are improved, but performance loss and energy consumption increase
Solution Approach 1:
The patent extracts only the essential information needed for data reconstruction by sending operation logs (sequences of operations) instead of complete data sets. This allows the receiving node to reconstruct the data state by applying the operations locally, significantly reducing transfer size while maintaining data availability.
Solution Approach 2:
Instead of transferring data directly from source to destination, the patent inverts the approach by transferring operations that transform data. The receiving node applies these operations to local data to achieve the same result, eliminating the need for large data transfers.
2Reliability
If data is transferred frequently and in large sizes between nodes, then data availability and completeness are improved, but energy consumption increases
Solution Approach 1:
The patent extracts only the essential information needed for data reconstruction by sending operation logs (sequences of operations) instead of complete data sets. This allows the receiving node to reconstruct the data state by applying the operations locally, significantly reducing transfer size while maintaining data availability.
Solution Approach 2:
The patent changes the parameter being transferred from raw data (large size) to operation descriptions (small size). By transforming the representation of data state changes, the system achieves the same functional result with minimal energy expenditure.
3Quantity of substance
If operation logs are compressed by removing inverse and idempotent operations, then transfer size is reduced, but processing complexity increases
Solution Approach 1:
The patent applies preliminary compression to operation logs by removing inverse operations (operations that undo each other) and idempotent operations (operations that have no net effect when applied multiple times). This preprocessing reduces the size of transferred data before transmission.
Solution Approach 2:
The system performs self-optimization by automatically identifying and removing redundant operations from the log. The compression algorithm handles the complexity internally, allowing the rest of the system to work with already-compressed operation sequences without needing to understand the compression logic.
Data Source
AI summary
Data transfer acceleration includes receiving, by a data transfer accelerator in a first node of a plurality of nodes, from a second node of the plurality of nodes, a request for data in a second state, wherein the second node stores an instance of the data in a first state; generating a message including one or more operations to transform the data from the first state to the second state; and sending the message to the second node in response to the request.