Neural Network Model Updating via Hard Label Transmission
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network model updating methods face challenges in minimizing communication costs and protecting personal information when transmitting large data sets between servers and terminal devices, which can lead to excessive communication costs and potential leakage of personal information.
Innovation Solution
A method where terminal devices train local models using local data sets and transmit only hard labels to a server, which updates a global model using these hard labels and ground truth labels, reducing the need for transmitting entire model parameters or soft labels, and allowing for model updates even when local and global models have different structures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If terminal devices transmit entire model parameters or soft labels to the server for model updating, then the server can update the global model with detailed information, but communication costs increase and personal information may be leaked
Solution Approach 1:
The patent extracts only the essential information (hard labels representing final inference results) from the complete model parameters or soft labels, and transmits only this extracted information to the server. This resolves the contradiction by maintaining model update capability while removing unnecessary data that increases communication cost and privacy risk.
Solution Approach 2:
The patent uses hard labels as simplified copies of the complete inference results. Instead of transmitting full model parameters or probability distributions (soft labels), it transmits compact hard label copies that capture the essential classification outcomes, reducing communication overhead while enabling effective global model updates.
2Loss of information
If terminal devices transmit soft labels containing probability values to the server, then the server can use detailed probability information for model updating, but the data size increases compared to hard labels
Solution Approach 1:
The patent extracts only the most critical information from soft labels - the final class prediction (hard label) - and transmits only this extracted information. This resolves the contradiction by maintaining sufficient inference information for model updates while dramatically reducing data size compared to transmitting complete probability distributions.
3Adaptability or versatility
If the global model and local model have different structures, then the system can be more flexible and adaptable, but updating the global model becomes more difficult
Solution Approach 1:
The patent introduces hard labels as an intermediary that bridges the gap between local models with different structures and the global model. By translating inference results from various local model structures into a unified hard label format, the system enables global model updates without requiring the local and global models to have matching structures, thus resolving the contradiction between flexibility and update complexity.
Data Source
AI summary
A method of updating a neural network model by a terminal device, includes training a local model using a local data set collected by a terminal device to generate a trained local model; receiving, from a server, an independent identically distributed (i.i.d.) global data set, the i.i.d. global data set being a data set sampled for each class in a plurality of predefined classes; implementing the trained local model by inputting the i.i.d. global data set and transmitting final inference results of the implemented trained local model to the server; and receiving, from the server, a global model updated based on the final inference results of the inference.