Dummy Prototypical Networks for Open-Set Rejection in Keyword Spotting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing keyword spotting systems face challenges in achieving accurate and efficient detection of keywords, particularly in real-time scenarios, due to limited user utterances for model training and the need to handle unexpected novel classes, leading to misclassification of open-set queries.
Innovation Solution
The use of a dummy prototypical network for few-shot open-set recognition (FSOSR) that generates prototype representations and dummy prototypes to classify query samples into known classes or an open-set class, improving the accuracy of keyword spotting by rejecting open-set queries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a keyword model is generated from a large number of utterances to improve detection performance, then the detection accuracy is improved, but the time required for model generation and user convenience deteriorates
Solution Approach 1:
The system performs preliminary action by generating a keyword model from a small number of utterances (few-shot learning) and then uses prototype representations to quickly classify new queries without requiring extensive retraining. The dummy prototype representation is pre-computed from support samples, enabling fast inference for open-set recognition.
Solution Approach 2:
The system creates a copy of the keyword model with prototype representations that capture the essential characteristics of known classes. This prototype-based model can quickly recognize patterns in query samples without requiring the full training data, enabling fast and accurate detection with minimal utterances.
2Loss of time
If a keyword model is generated from a small number of utterances to improve user convenience, then the time required for model generation is reduced, but the detection performance deteriorates
Solution Approach 1:
The system extracts the essential information from the limited support samples by computing prototype representations. These prototypes capture the key characteristics of each known class, allowing the system to achieve good detection performance even with few-shot learning by focusing on the most informative features from the limited data.
Solution Approach 2:
The prototype representations serve as an intermediary between the limited support samples and the query samples. They mediate the classification process by providing a compact summary of known classes that can be efficiently compared against query samples, enabling accurate detection despite the small number of training utterances.
3Speed
If existing keyword spotting systems are used to detect keywords in real-time, then the system responds quickly, but misclassification of open-set queries occurs
Solution Approach 1:
The system segments the classification task into two parts: (1) computing prototype representations from support samples for known classes, and (2) comparing query samples against these prototypes to determine classification. This segmentation enables efficient real-time processing by avoiding the need to reprocess all training data for each query.
Solution Approach 2:
The system replaces the mechanical system of comparing query samples against all training data with a prototype-based comparison system. This substitution reduces computational complexity and enables faster response times while maintaining high classification accuracy through the use of dummy prototype representations for open-set recognition.
Data Source
AI summary
Systems and techniques are provided for processing audio data. For example, a dummy prototypical network may be used to perform few-shot open-set keyword spotting (FSOS-KWS). A process can include determining one or more prototype representations based on a plurality of support samples associated with one or more classes. Each prototype representation may be associated with one of the class(es). A dummy prototype representation can be determined in a same learned metric space as the prototype representations. One or more distance metrics can be determined for each query sample of one or more query samples. The distance metrics may be based on the prototype representations and the dummy prototype representation. Each query sample can be classified based on the distance metrics. Each query sample may be classified into one of the class(es) associated with the prototype representations or into an open-set class associated with the dummy prototype representation.