Segment Similarity Data Transfer for Deduplicated Transmission
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As data quantities increase, existing data transmission methods struggle to efficiently transmit data between local and remote devices over networks, leading to slower transmission speeds and higher storage requirements.
Innovation Solution
A method and system that divide original data into segments, calculate similarity values, and transmit these values to determine which segments exist on the target device, allowing for delta compression and deduplication to reduce data transmission quantity, with the target device rehydrating the data upon receipt.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is transmitted in its original form, then data integrity is maintained, but data transmission quantity increases
Solution Approach 1:
The patent performs preliminary actions by calculating similarity values and determining which target segments already exist on the target device before actual data transmission. This allows the system to identify and exclude redundant data segments from transmission, reducing data transmission quantity while maintaining data integrity through selective transmission of only necessary segments.
Solution Approach 2:
The patent extracts and transmits only the necessary data segments that are not already present on the target device. By using similarity calculation to identify existing segments and removing these from the transmission set, the system reduces data transmission quantity while ensuring that only missing or updated data is transmitted, thus maintaining data integrity.
2Productivity
If all data segments are transmitted, then complete data is received, but transmission time increases
Solution Approach 1:
The patent performs preliminary similarity calculations and segment comparisons before transmission to identify which data segments need to be transmitted. This preliminary action enables the system to exclude already-existing segments from transmission, significantly reducing transmission time while ensuring complete data reconstruction at the target device through selective transmission of only necessary segments.
3Quantity of substance
If similarity calculation is performed for all segments, then transmission optimization is achieved, but computational complexity increases
Solution Approach 1:
The patent divides data into multiple segments and performs similarity calculations on individual segments rather than on the entire dataset. This segmentation approach reduces computational complexity by breaking down the large-scale comparison task into smaller, manageable segment-level operations, while still achieving effective transmission optimization through segment-level deduplication.
Solution Approach 2:
The patent performs similarity calculation only for necessary segments that may require transmission, rather than calculating for all segments uniformly. By using partial action on selected segments based on transmission needs, the system achieves transmission optimization while reducing overall computational complexity compared to exhaustive full-segment analysis.
Data Source
AI summary
A method and a system for transmitting data are provided. In a source apparatus, original data is divided into a plurality of source segments, a similarity calculation is performed for each of the source segments to obtain a similarity set, and the similarity set is transmitted to a target apparatus. In the target apparatus, whether a target segment corresponding to the source segment exists in the target apparatus is determined according to the similarity set to obtain a comparison result, and the comparison result is transmitted to the source apparatus. In the source apparatus, after the original data is dehydrated according to the comparison result to obtain dehydration data, the dehydration data is transmitted to the target apparatus. In the target apparatus, the dehydration data is rehydrated to the original data.