Random Linear Code Synchronization for Large Distributed Data Sets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data synchronization methods in distributed computing systems are resource-intensive and inefficient, particularly in terms of computational burden and energy consumption, especially when dealing with large data sets and error-prone communication channels, and they often require significant computational resources and energy on battery-operated devices like mobile phones.
Innovation Solution
The method employs random linear codes for data synchronization, using a divide-and-solve approach with set partitioning to reduce computational complexity and energy consumption, and incorporates error correction techniques to ensure robustness and fault tolerance, particularly by using a hierarchical tree structure for encoding and decoding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional data synchronization methods are used, then data consistency is achieved, but computational burden and energy consumption increase significantly
Solution Approach 1:
The patent divides the data-set into multiple blocks and processes them independently. Each block is handled separately through encoding and transmission, which reduces the overall computational burden compared to processing the entire data-set as a single unit. This segmentation allows for more efficient resource utilization and lower energy consumption during synchronization operations.
2Reliability
If frequent synchronization is performed to maintain data consistency, then data accuracy is improved, but communication and computation resources are excessively consumed
Solution Approach 1:
The patent extracts and transmits only the essential information needed for synchronization - specifically, encoded representations of data blocks and their corresponding hash values. By transmitting only this extracted essential information rather than complete data-sets, the communication overhead is significantly reduced, allowing for more frequent synchronization without excessive resource consumption.
3Reliability
If complete data transmission is used for synchronization, then data accuracy is ensured, but data exchange volume increases
Solution Approach 1:
The patent uses hash values as compact representations or 'copies' of the actual data blocks. Instead of transmitting complete data blocks for verification, the system transmits only the hash values which serve as efficient proxies. This copying approach maintains data accuracy through hash verification while dramatically reducing the volume of data exchanged during synchronization.
4Reliability
If robust error correction is implemented for fault-tolerant synchronization, then reliability over error-prone channels is improved, but computational complexity increases
Solution Approach 1:
The patent applies encoding to data blocks before transmission, preparing them in advance for potential errors. This preliminary encoding action embeds error-correcting capabilities into the data structure beforehand, allowing the receiving end to detect and correct errors without requiring complex real-time computation during the synchronization process itself.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method and a distributed computing system (1) are provided to use a forward error correction code for both data synchronization as well as an error-resistant data transmission during the data synchronization process. In particular, a random linear code is used which is traditionally used as an error correction code. In order to synchronize large data-sets by using random linear codes a set partitioning is performed on the large data sets to divide each data-set into a plurality of data-subsets. Therefore, the distributed computing system (1) is made scalable by dividing the data synchronization issue into many smaller problems. This results in significantly lower computational complexity at the decoding communication device (20), allowing the data synchronization to complete much faster (lesser computation time of the system) and also save communication between the two synchronizing communication devices (10, 20).