Intent Classifier Training via Boolean Negation Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Natural Language Processing (NLP) models face challenges in accurately identifying user intents due to poorly constructed training datasets, which often lack sufficient positive and negative examples, leading to nonsensical associations between utterances and intents.
Innovation Solution
A method for class balancing in intent authoring using search, where positive and negative example utterances are extracted from conversation logs using Boolean operators to build balanced training datasets, ensuring accurate intent classification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional training datasets are used for NLP model training, then the model can be trained with available data, but the dataset lacks sufficient positive and negative examples leading to inaccurate intent classification
Solution Approach 1:
The system performs preliminary actions by automatically generating synthetic training examples (both positive and negative) before model training. It uses the provided positive example to generate additional positive examples through paraphrasing and template expansion, and creates negative examples by manipulating the positive example (swapping entities, negating intent). This preliminary generation of balanced training data resolves the contradiction by ensuring sufficient quantity of diverse examples while maintaining classification accuracy.
Solution Approach 2:
The system employs self-service by using the limited positive example input to automatically generate its own training dataset. The generator uses the single positive example to create multiple positive variations through paraphrasing, template expansion, and back-translation, while also generating negative examples through entity swapping and intent negation. This self-service approach resolves the contradiction by transforming a single example into a comprehensive balanced dataset without requiring external manual annotation.
2Reliability
If manual dataset construction is performed to ensure balanced positive and negative examples, then classification accuracy improves, but the process becomes time-consuming and resource-intensive
Solution Approach 1:
The system performs self-service by automatically generating balanced training datasets without human intervention. It takes a single positive example as input and autonomously generates multiple positive examples through paraphrasing, template expansion, and back-translation, while simultaneously creating negative examples through entity swapping and intent negation. This automated self-service approach resolves the contradiction by achieving dataset balance quality that would otherwise require extensive manual effort, thereby eliminating time loss associated with manual dataset construction.
Solution Approach 2:
The system uses copying by creating multiple variations of the single positive example through paraphrasing, template expansion, and back-translation. It copies the core intent structure while varying the surface form to generate diverse positive examples. For negative examples, it copies the positive example structure and modifies it through entity swapping and intent negation. This copying approach resolves the contradiction by efficiently generating balanced datasets through systematic replication and transformation rather than time-consuming manual creation.
3Adaptability or versatility
If diverse training examples are generated through multiple methods, then dataset comprehensiveness improves, but the complexity of the generation process increases
Solution Approach 1:
The system applies segmentation by dividing the training example generation process into distinct modular components: positive example generation (paraphrasing, template expansion, back-translation) and negative example generation (entity swapping, intent negation). Each module handles a specific aspect of dataset generation independently. This segmentation resolves the contradiction by organizing the complex generation process into manageable, independent modules that can be executed systematically, thereby achieving diverse and comprehensive datasets without overwhelming process complexity.
Solution Approach 2:
The system employs universality by designing a multi-functional generator that performs multiple operations using unified principles. The same generator handles both positive and negative example generation, uses paraphrasing for both types, applies template expansion universally, and employs back-translation for diversity. This universal approach resolves the contradiction by achieving dataset diversity through consistent application of multiple generation methods across different example types, rather than requiring separate complex processes for each.
Data Source
AI summary
Embodiments provide for class balancing for intent authoring using search via: receiving a positive example of an utterance associated with an intent, building an in-intent pool of utterances from a conversation log using the positive example in a first search query of the conversation log; adding the in-intent pool of utterances as a positive class to a training dataset; applying Boolean operators to negate the positive example to form a complement example; building an out-intent pool of utterances from the conversation log using the complement example in a first search query of the conversation log; and adding the out-intent pool of utterances as a complement class to the training dataset. The training dataset may be balanced to include a predefined ratio of positive and complement examples. The training dataset may be used to train or retrain an intent classifier.