Selective Data Deduplication via Dynamic Hash Function Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing communication systems face inefficiencies in data transfer and integrity checking, particularly when network utilization exceeds thresholds, leading to increased data redundancy and processing burdens.
Innovation Solution
Implementing a method that uses cryptographic hash functions for selective deduplication and integrity checking by switching between hash functions based on CPU utilization thresholds, allowing for efficient data transfer and integrity verification without redundant data transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional communication protocols transmit all data packages without deduplication, then data integrity checking is maintained, but network bandwidth is wasted and data transfer volume increases
Solution Approach 1:
Instead of transmitting the actual data package, the system transmits only the hash digest (a compact representation) of the data package. The receiving device can then verify integrity by computing the hash of the received data and comparing it with the transmitted hash digest, avoiding redundant data transmission while maintaining integrity verification capability
Solution Approach 2:
The system extracts only the essential information (hash digest) from the data package for transmission, separating the integrity verification function from the data transfer function. This allows the receiving device to verify integrity without receiving the full data package, reducing network bandwidth usage
2Device complexity
If a single hash function is used for all data packages, then implementation is simple, but CPU resources are not optimized under varying network conditions
Solution Approach 1:
The system dynamically selects which hash function to use based on real-time network conditions and CPU utilization metrics. When network utilization is high, the system switches to a more efficient hash function that requires less CPU processing, optimizing resource allocation under varying operational conditions
Solution Approach 2:
The system changes the parameter of hash function selection based on monitored network and CPU conditions. By adjusting which hash function is applied (e.g., switching between different cryptographic hash algorithms), the system optimizes the balance between security requirements and processing resource consumption
Data Source
AI summary
An approach for managing a data package is provided. Network utilization is determined to exceed a threshold. A sender computer determines a hash digest of the data package by using a hash function selected based on central processing unit utilization. If the hash digest is in a sender hash table, then without sending the data package, the sender computer sends the hash digest and an index referring to the hash digest so that a recipient computer can use the index to locate a matching hash digest and the data package in a recipient hash table. If the hash digest is not in the sender hash table, then the sender computer adds the data package and the hash digest to the sender hash table and sends the data package and the hash digest to the second computer to check the integrity of the data package based on the hash digest.