Client Data Digest for Server Model Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data mining processes in client-server architectures consume excessive resources such as bandwidth, traffic, and computing power due to continuous data transmission from clients to servers, leading to high load pressure and inefficient resource utilization.
Innovation Solution
The client generates a data digest summarizing the stored data set and sends it to the server, which determines a data selection policy and parameter, allowing the client to selectively transmit a data subset for model updates, thereby reducing the amount of data transmitted and resource consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the client constantly transmits new data to the server for model updates, then the prediction model can be continuously updated to improve accuracy, but the resource consumption (bandwidth, traffic, computing power) increases significantly
Solution Approach 1:
The patent extracts only the essential information from the original data by generating a data digest (hash value) that represents the key characteristics of the data set. Instead of transmitting all raw data, only the compact digest is sent to the server, which then determines the data selection policy based on this digest. This extraction principle reduces transmission volume while preserving the essential information needed for model update decisions.
Solution Approach 2:
The patent segments the data transmission process into multiple stages: first transmitting a compact data digest, then based on server feedback, selectively transmitting only the necessary data subset. This segmentation allows the system to avoid transmitting unnecessary data while ensuring the model receives the essential information for accurate updates, thereby reducing overall resource consumption.
2Reliability
If the client transmits all data to the server, then the server can process complete information for model updates, but the data transmission volume and server processing load become excessive
Solution Approach 1:
The patent applies preliminary action by having the client generate a data digest before transmission and send it to the server in advance. The server uses this digest to pre-determine the data selection policy and provides feedback to the client. This preliminary step allows the system to identify and transmit only the necessary data subset, ensuring complete information for model updates while minimizing transmission volume.
Solution Approach 2:
The data digest acts as an intermediary between the client's data set and the server's model update process. Instead of directly transmitting all raw data, the digest serves as a compact representation that enables the server to determine the appropriate data selection policy. This intermediary mechanism preserves the essential information needed for reliable model updates while significantly reducing the quantity of data that must be transmitted.
3Loss of energy
If the client selects and transmits data subsets, then the transmission volume is reduced, but the selection process requires additional processing overhead
Solution Approach 1:
The patent applies parameter changes by transforming the data set into a different parameter representation (data digest/hash value) that captures the essential characteristics in a compact form. This parameter transformation simplifies the subsequent data selection process, as the server can determine the data selection policy based on the digest without needing to analyze the entire data set, thereby reducing both transmission consumption and selection complexity.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Embodiments of the present invention disclose a data processing method and a related device, and a system. The method includes: a client generates a data digest for a stored data set and sends the data digest to a server; the server determines feedback information for the client according to the data digest and sends the feedback information to the client; when the feedback information received by the client includes a target data selection policy and a target policy parameter, the client may determine a data subset from the data set according to the target data selection policy and the target policy parameter and send the data subset to the server; and the server may establish a prediction model or update an existing prediction model according to the data subset. By implementing the embodiments of the present invention, an amount of data transmitted by the client to the server can be controlled. Therefore, resource consumption in a data processing process is effectively reduced, and in addition, representative data can be accurately and conveniently selected, so as to obtain a more accurate prediction model.