Decentralized Recommendation Model Training with Secure Multi-Party Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing recommendation systems typically require centralized data collection, which can compromise user privacy and are inefficient, as they often involve linear and sequential processing across clients, leading to high computational costs and privacy risks.
Innovation Solution
A decentralized approach using blind learning and secure multi-party computation (MPC) for matrix aggregation, where a neural network is split between clients and a server, allowing parallel and independent processing, and secure sharing of gradients to maintain privacy and improve efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If centralized data collection is used for training recommendation systems, then training efficiency is improved, but user privacy is compromised
Solution Approach 1:
The neural network is divided into multiple parts distributed across different clients. Each client holds and processes only its local data and corresponding network portion, eliminating the need to centralize raw data while still enabling collaborative training through shared gradients and model updates.
Solution Approach 2:
A secure multi-party computation framework acts as an intermediary between clients. This framework enables coordinated training and model aggregation without requiring clients to share their raw data, using cryptographic protocols to ensure privacy preservation during the training process.
2Device complexity
If sequential processing across clients is used, then coordination is simplified, but computational costs increase
Solution Approach 1:
Multiple clients process and train their local network portions simultaneously in parallel rather than sequentially. This parallel processing combines the computational work of multiple clients into concurrent operations, significantly reducing total training time and computational costs while maintaining coordination through the distributed framework.
3Object-affected harmful factors
If decentralized learning methods are used to protect privacy, then user privacy is improved, but training efficiency deteriorates
Solution Approach 1:
The system dynamically adjusts the split of the neural network into different portions assigned to each client based on factors like data size, computational capability, and training requirements. This dynamic allocation optimizes the balance between privacy protection and training efficiency by adapting to varying conditions across clients.
Solution Approach 2:
The framework changes key parameters such as the number of network portions, the size of each portion, and communication frequencies to optimize training efficiency while maintaining privacy. By adjusting these parameters, the system achieves efficient distributed training without compromising the privacy benefits of decentralized learning.
Data Source
AI summary
A system and method are disclosed for training a recommendation system. The method includes initiating, at a server device, an item-vector matrix V, wherein the item-vector matrix V includes a value m related to a total number of items across one or more client devices and a value d representing a hidden dimension, transmitting the item-vector matrix V to each client device, wherein each client device trains a local matrix factorization model using a respective user vector U and the item-vector matrix V to generate a respective set of gradients on each respective client device, receiving, via a secure multi-party compute protocol, and from each client device, the respective set of gradients, updating the item-vector matrix V using the respective set of gradients from each client device to generate an updated item-vector matrix V and downloading the updated item-vector matrix V to at least one client device.