OOS Sentence Generation for Voice Assistant Intent Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems fail to accurately identify and reject out-of-service (OOS) sentences within a target service, leading to unintended function execution, as they lack effective methods to differentiate between in-service (IS) and OOS inputs.
Innovation Solution
A method and apparatus for generating OOS sentences using a model trainer and sentence generator that train neural network models to convert target utterance templates into similar sentences, allowing for the identification of OOS intents by allocating an OOS label to similar sentences that request functions not available in the target service.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the system uses a classifier to determine intent of in-service inputs, then intent determination accuracy for in-service sentences is improved, but the system cannot accurately differentiate and reject out-of-service inputs
Solution Approach 1:
The patent segments the training data into two distinct categories: in-service (IS) sentences and out-of-service (OOS) sentences. The IS sentences are used to train the classifier for intent determination, while the OOS sentences are used to train a separate model for identifying serviceable inputs. This segmentation allows the system to specialize each model for its specific function, improving both intent determination accuracy and OOS rejection capability without interference between the two functions.
Solution Approach 2:
The patent introduces an intermediate OOS sentence generation system that creates synthetic OOS training data by modifying target utterance templates. This intermediate component generates realistic OOS sentences that preserve linguistic patterns while representing invalid service requests, which then serves as training data for the OOS detection model. This intermediary approach solves the problem of lacking sufficient real OOS examples for training.
2Productivity
If the system trains on available in-service data, then the classifier can determine intent for known services, but it fails to identify out-of-service sentences that request unavailable functions
Solution Approach 1:
The patent performs preliminary action by generating OOS training data in advance through systematic modification of target utterance templates. Before the actual classification task, the system creates a comprehensive dataset of OOS sentences by replacing nouns, verbs, and other components in valid service templates with inappropriate alternatives. This pre-generated OOS data is then used to train the detection model, enabling accurate OOS identification during runtime without requiring real-time analysis of unavailable functions.
Solution Approach 2:
The patent uses copying by creating synthetic OOS sentences that replicate the structural and linguistic patterns of valid IS sentences. Instead of relying on scarce real OOS examples, the system copies the template structure of serviceable utterances and fills them with modified components that represent invalid requests. These copied sentences serve as realistic training examples that teach the model to recognize OOS patterns while maintaining the same linguistic characteristics as valid inputs.
3Ease of operation
If the threshold value for restoration error is fixed, then the auto-encoder model can classify inputs as IS or OOS, but the threshold may not adapt to varying input characteristics and training data
Solution Approach 1:
The patent applies dynamics by making the restoration error threshold adaptive rather than fixed. The threshold value is dynamically determined based on the specific input characteristics and the training data being used. The system calculates appropriate threshold values that adapt to varying input distributions and service domains, allowing the auto-encoder model to maintain accurate classification performance across different scenarios without requiring manual threshold adjustment for each case.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An out-of-service (OOS) sentence generating method includes: training models based on a target utterance template of a target service and a target sentence generated from the target utterance template; generating a similar utterance template that is similar to the target utterance template based on a trained model, among the trained models, and a sentence generated from an utterance template of another service; and generating a similar sentence that is similar to the target sentence based on another trained model, among the trained models, and the similar utterance template.