Client Hash Value Transmission for Duplicate Data Elimination
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In the client-server architecture for machine learning and deep learning tasks, a significant amount of data is duplicated during transmission, leading to inefficient use of network bandwidth and potential network congestion, which affects the performance of dedicated processors.
Innovation Solution
A method where a client determines if data has been previously transmitted by checking a set of hash values, and if so, only transmits the hash value instead of the original data, while adding the hash value to the set for future reference, thereby avoiding duplicate data transmission and reducing the need for additional remote procedure calls.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If original data is transmitted every time in client-server architecture, then data completeness is ensured, but network bandwidth is occupied excessively and duplicated data transmission occurs
Solution Approach 1:
The patent uses hash values as copies of the original data to identify and avoid transmitting duplicate data. Instead of transmitting the full original data every time, the system transmits hash values to check for duplicates, and only transmits original data when necessary, thus reducing network bandwidth consumption while maintaining data completeness
Solution Approach 2:
The patent performs preliminary hash value calculation and comparison before data transmission. By checking whether the hash value of the original data exists in the set of hash values before transmitting, the system prevents duplicate transmission in advance, reducing network bandwidth consumption without compromising data completeness
2Reliability
If data transmission frequency is increased to ensure data availability, then data completeness is maintained, but waiting time for dedicated processors increases
Solution Approach 1:
The patent performs preliminary hash value checking before data transmission to determine whether data needs to be sent. This preliminary action avoids unnecessary transmission waits and reduces the time dedicated processors spend waiting for data, while still ensuring data availability when needed
3Quantity of substance
If hash value checking mechanism is implemented, then duplicate data transmission is reduced, but device complexity increases
Solution Approach 1:
The patent uses hash values as compact copies of data identifiers. By maintaining a set of hash values and comparing hashes before transmission, the system reduces data transmission volume without requiring complex data comparison mechanisms, thus managing system complexity while achieving duplicate reduction
Data Source
AI summary
Embodiments of the present disclosure relate to methods and an electronic device for transmitting and receiving data. The data transmission method includes: determining a hash value of original data to be transmitted; determining whether the hash value exist in a predetermined set of hash values; in response to the hash value being present in the set of hash values, transmitting the hash value, rather than the original data, to a server; and in response to the hash value being absent from the set of hash values, transmitting the original data to the server; and adding the hash value to the set of hash values. The embodiments of the present disclosure can avoid transmitting duplicated data between a client and a server, and it is not required to add extra remote procedure calling commands between the client and the server.