On-Device Speaker Verification Model Personalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current speaker verification models struggle to identify 'challenging' negative samples for personalization on-device while complying with privacy laws, leading to suboptimal accuracy and increased memory requirements.
Innovation Solution
A method that involves a server identifying a group of users with similar voices to a specific user, selecting and transmitting these negative audio samples to the user's device for personalization, using a classifier to determine voice characteristics and update the central server's classifier through federated learning, thereby enhancing model accuracy and reducing memory needs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If challenging negative samples are stored on-device for personalisation, then model accuracy is improved, but memory requirements increase
Solution Approach 1:
The patent extracts only the most challenging negative samples from the full dataset and stores them on-device, while keeping the complete dataset on the server. This selective extraction allows the on-device model to access challenging samples for personalisation without storing the entire dataset, thus improving accuracy while limiting memory usage.
Solution Approach 2:
The patent segments the negative samples into two groups: challenging samples that are transferred to the on-device for personalisation, and non-challenging samples that remain on the server. This segmentation allows the system to optimise memory usage by only storing essential samples locally while maintaining access to the full dataset remotely.
2Measurement precision
If challenging negative samples are selected for personalisation, then model accuracy is improved, but difficulty in identifying suitable samples increases
Solution Approach 1:
The patent uses a feedback mechanism where the server evaluates which negative samples are most challenging for each user's model and selectively transfers only those samples to the on-device. This feedback loop allows the system to automatically identify and select challenging samples without manual intervention, improving accuracy while managing the complexity of sample selection.
3Measurement precision
If audio samples are stored on-device for personalisation, then personalisation accuracy is improved, but data security risks increase
Solution Approach 1:
The patent extracts only the essential challenging negative samples needed for personalisation and transfers them to the on-device, while keeping the majority of the audio data on the secure server. This selective extraction minimises the amount of sensitive data stored on potentially less secure end-user devices, thereby reducing data security risks while maintaining personalisation effectiveness.
Data Source
AI summary
Broadly speaking, embodiments of the present techniques provide a method for personalising a trained speaker verification machine learning, ML, model for a specific user, on-device (i.e. on the end user device which is going to be used to run the personalised ML model). Advantageously, the present techniques improve the personalisation of the ML model on-device without requiring large volumes of data to be stored on the device.