Federated Learning for Private Wakeword Detection Retraining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Speech processing systems face issues with false-positive and false-negative detections of wakewords, leading to diminished usability and user frustration, and existing solutions often require sharing sensitive user data, compromising privacy.
Innovation Solution
A federated learning approach is employed to update machine learning models on devices using gradient data, where devices share model parameter updates rather than raw audio data, and a centralized system confirms detections to improve model accuracy through a combination of local and remote training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If speech processing systems share raw audio data for model training, then model accuracy is improved, but user privacy is compromised
Solution Approach 1:
The patent extracts only the essential training information (gradient updates) from the raw audio data, separating the useful model-improving signal from the privacy-sensitive raw data. Devices compute gradients locally and share only these aggregated updates with the centralized server, eliminating the need to transmit or store actual audio recordings while still enabling model training.
Solution Approach 2:
The patent introduces gradient updates as an intermediary between raw audio data and model training. Instead of directly sharing audio data, the system uses gradient computations as a mediator that transforms private audio inputs into privacy-preserving training signals that can be aggregated and used for improving the wakeword detection model.
2Object-affected harmful factors
If devices process audio data locally to preserve privacy, then user privacy is protected, but model accuracy deteriorates due to limited local data
Solution Approach 1:
The patent merges the computational capabilities of distributed devices with the aggregating power of a centralized server. Local devices perform privacy-preserving gradient computations on their own audio data, while the centralized server aggregates these gradients from multiple devices to create an improved global model, combining the benefits of both distributed privacy protection and centralized data pooling.
Solution Approach 2:
The patent segments the model training process into two distinct phases: local gradient computation performed independently on each device to preserve privacy, and centralized aggregation of these gradients to improve the global model. This segmentation allows each component to operate in its optimal environment while contributing to the overall training objective.
3Measurement precision
If centralized systems process all audio data for training, then model accuracy is improved, but system complexity and data transmission requirements increase
Solution Approach 1:
The patent segments the training workload between edge devices and centralized servers, with each performing specialized computations. This segmentation reduces the complexity burden on any single component while enabling the system to achieve high model accuracy through coordinated distributed processing.
Data Source
AI summary
A system performs federated learning and retraining of a machine learning model used for processing audio detected by a user device. The system uses both gradient data (which may correspond to false-rejects) and audio data (which may correspond to false-positives) received from devices. The system may also use a teacher model to produce labels for data in an automated fashion, thus allowing retraining to happen in an unsupervised manner.