Federated Feature Extraction for Privacy-Preserving Model Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning models face challenges in effectively fusing diverse training data from multiple companies while ensuring data privacy and security, which hinders the improvement of click-through rates and conversion rates in advertising services.
Innovation Solution
A multi-model training method based on federated feature extraction, where tree models are trained collaboratively with encrypted data transmission, and feature importance evaluation is performed to assign weights to feature columns, enabling the screening and training of linear models with enhanced data fusion and privacy protection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data from multiple companies is fused to train recommendation engines, then data diversity and model robustness are improved, but data privacy and security are compromised
Solution Approach 1:
The patent segments the training process into two distinct phases: a pre-training phase where tree models are trained on local data without sharing, and a fine-tuning phase where only model parameters are shared. This segmentation allows data diversity to be leveraged through parameter sharing while preventing raw data from leaving the premises, thus resolving the contradiction between data diversity and privacy protection.
Solution Approach 2:
The patent introduces model parameters as an intermediary between companies. Instead of directly sharing sensitive raw data, companies share only the extracted features and model parameters through a federated learning framework. This intermediary mechanism enables collaborative training while maintaining data confidentiality, as the intermediate representations do not contain direct access to original private data.
2Reliability
If encrypted data transmission is used for collaboration, then data security is improved, but training efficiency deteriorates
Solution Approach 1:
The patent extracts only the essential model parameters and feature representations for transmission during federated training, leaving the bulk of raw data local to each company. By extracting and sharing only the necessary compressed representations rather than transmitting full encrypted datasets, the system maintains strong security while significantly reducing communication overhead and improving training efficiency.
3Measurement precision
If feature importance evaluation is performed to screen features, then model precision is improved, but computational complexity increases
Solution Approach 1:
The patent performs feature importance evaluation and screening as a preliminary step before the main model training process. By pre-identifying and selecting the most important features using tree-based importance metrics, the system reduces the dimensionality of the feature space early on. This preliminary action simplifies subsequent training computations while ensuring that only the most relevant features are processed, thus improving precision without excessive complexity.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
A method includes training, in collaboration with corresponding ones of a plurality of collaborators, corresponding ones of a plurality of tree models based on corresponding data of user samples shared with the corresponding ones of the plurality of collaborators; performing feature importance evaluation on the trained tree models for assigning respective weights to feature columns generated by respective ones of the tree models; in response to a determination that a linear model is to be trained by a first collaborator, inputting data of a first user sample shared with the first collaborator to the tree model corresponding to the first collaborator and the tree models corresponding to a second collaborator, to obtain a plurality of one-hot encoded feature columns; and screening the obtained feature columns based on the respective weights and training the linear model according to the screened feature columns and the data of the first user sample.