Incremental IBLT Cell Transfer for Efficient Data Reconciliation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data reconciliation methods, such as those using IBLTs, require an upper limit to the set difference which is often not available, leading to inefficient communication and bandwidth usage due to oversizing or underestimation of IBLTs, and they lack an incremental approach to reduce communication effort.
Innovation Solution
An incremental method using multi-edge type IBLTs where cells are transmitted sequentially, allowing the receiver to attempt decoding after each transmission, with an optional initial communication round for parameter adaptation, and using a modified recovery algorithm to optimize decoding efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire IBLT data structure is transmitted from the first user terminal to the second user terminal, then the data set reconciliation can be performed, but the communication effort and bandwidth usage increase unnecessarily
Solution Approach 1:
The IBLT data structure is divided into multiple individual cells, which are transmitted sequentially rather than all at once. The transmitter sends cells one by one, and the receiver attempts decoding after each cell is received, allowing the process to stop as soon as successful reconciliation is achieved.
Solution Approach 2:
Instead of transmitting the complete IBLT data structure, only the necessary number of cells are transmitted to achieve successful decoding. The receiver attempts decoding after receiving each cell, and transmission stops as soon as the minimum required cells are received, avoiding unnecessary communication overhead.
2Productivity
If an upper limit to the set difference is used to size the IBLT, then the IBLT can be optimized for the expected difference, but the IBLT may be oversized or underestimated when the actual difference deviates from the limit
Solution Approach 1:
The system adapts to the actual set difference dynamically during the transmission process. Instead of relying on a pre-set upper limit that may be inaccurate, the incremental transmission allows the receiver to attempt decoding after each cell and adjust the process accordingly, making the system robust to deviations from estimated parameters.
3Reliability
If all IBLT cells are transmitted before attempting decoding, then complete data is available for recovery, but the communication effort is wasted when fewer cells would have been sufficient
Solution Approach 1:
The receiver provides feedback by attempting decoding after each cell is received. When successful decoding is achieved, the receiver can signal the transmitter to stop transmission. This feedback mechanism ensures that transmission stops as soon as the minimum necessary cells are received, avoiding unnecessary communication time.
Data Source
AI summary
The invention relates to a method for transferring an Invertible Bloom Lookup Table (IBLT) data structure from a first user terminal to a second user terminal. The first user terminal and the second user terminal are within a common network and each have a computing unit, a memory unit and a communication unit. The method includes the first user terminal splits the IBLT data structure into individual data packages, wherein the individual data packages have at least one cell of the IBLT data structure; and the first user terminal successively transfers the individual data packages of the IBLT data structure to the second user terminal until a predefined termination criterion has been reached.