Federated Learning Clustering for User Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing user attribute prediction systems face challenges in maintaining accuracy due to data distribution differences across devices and the lack of comprehensive feature extraction, particularly with unstructured data, leading to reduced prediction accuracy and privacy concerns in data handling.
Innovation Solution
A prediction model training method that involves federated learning with clustering and hierarchical aggregation, where devices are grouped based on user features and processing capabilities, enabling global and intra-group parameter aggregation to update model parameters, and incorporating unstructured data features for improved prediction accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If user data is collected and stored on the server for prediction, then prediction accuracy is improved, but user privacy security deteriorates
Solution Approach 1:
The patent extracts the prediction model from the server and deploys it to user devices. The model parameters are updated through federated learning where only model updates (not raw user data) are transmitted to the server. This extracts the harmful element (centralized data storage) while preserving the useful element (prediction accuracy).
Solution Approach 2:
The patent introduces model parameters as an intermediary between user data and the server. Instead of directly transmitting user data to the server for prediction, the system uses model parameters that are updated through aggregated learning from multiple devices. This intermediary mechanism enables accurate prediction without exposing raw user privacy data.
2Ease of operation
If federated learning is used without device grouping, then implementation simplicity is improved, but prediction accuracy deteriorates due to data distribution differences
Solution Approach 1:
The patent segments the federated learning process into two levels: global aggregation across all devices and intra-group aggregation within device groups. Devices are grouped based on similarities in user features and processing capabilities. This segmentation allows the system to handle data distribution differences by creating homogeneous groups while maintaining overall system simplicity through automated clustering.
Solution Approach 2:
The patent applies local quality by performing intra-group federated aggregation separately for each device group based on their specific characteristics. Each group receives tailored model updates that consider their local data distribution and device capabilities, while the global aggregation ensures consistency across the entire system. This localized approach improves accuracy without significantly complicating implementation.
3Productivity
If only structured data is used for feature extraction, then processing efficiency is improved, but prediction accuracy deteriorates due to lack of unstructured data features
Solution Approach 1:
The patent uses composite materials analogy by combining multiple feature types (structured and unstructured data features) into a comprehensive feature representation. The feature extraction process integrates both easily processable structured data and richer unstructured data, creating a composite feature set that leverages the advantages of both data types for improved prediction accuracy while maintaining reasonable processing efficiency.
Data Source
AI summary
A prediction model training method includes transmitting a model to be trained by a plurality of training devices, the model to be trained including feature extraction layers and prediction layers, classifying the plurality of training devices into at least one group based on extracted user features, receiving, from the plurality of training devices, model parameters including first parameters corresponding to the feature extraction layers and second parameters corresponding to the prediction layers, performing global federated aggregation based on the first parameters, performing intra-group federated aggregation for each of the at least one group, based on the second parameters of one or more of the plurality of training devices in a respective group, and transmitting, to the plurality of training devices, the global federated aggregation result and the intra-group federated aggregation result.