Cross-Lingual Intent Classification With Semantic Similarity and OOD Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models for intent classification in conversational systems require extensive training on large datasets in multiple languages and domains, leading to increased training time and computational overhead.
Innovation Solution
An instance-based semantic similarity model using a pre-trained multilingual language model and a weighted K-nearest neighbor classifier to classify intents across multiple languages with reduced training data, incorporating an OOD threshold to identify out-of-domain utterances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If separate machine learning models are trained per language and domain, then service specificity is improved, but training time and computational overhead increase
Solution Approach 1:
The patent combines multiple language models into a single unified model that can handle multiple languages and domains simultaneously. The system uses a shared encoder architecture with language-specific and domain-specific adaptation layers, allowing one model to serve multiple purposes that traditionally required separate models, thereby reducing training time while maintaining service specificity.
Solution Approach 2:
The patent creates a universal machine learning model capable of functioning across multiple languages and domains through a single system. The model employs multi-task learning with shared representations that can be adapted to different languages and domains, making the system multi-functional and eliminating the need for separate specialized models.
2Reliability
If separate machine learning models are trained per language and domain, then service specificity is improved, but computational overhead increases
Solution Approach 1:
The patent merges multiple computational models into a single unified architecture, reducing the total computational resources required. By sharing encoder layers and using efficient adaptation mechanisms, the system decreases memory usage and processing power requirements compared to maintaining separate models for each language and domain.
Solution Approach 2:
The universal model performs multiple functions (supporting multiple languages and domains) within a single computational framework, reducing overall computational overhead. The shared representations and efficient inference mechanisms allow the system to handle diverse tasks without requiring proportional increases in computational resources.
3Measurement precision
If extensive training data in multiple languages is used, then classification accuracy is improved, but data requirements and training complexity increase
Solution Approach 1:
The patent applies pre-training on large multilingual corpora before fine-tuning on specific domain data. This preliminary action allows the model to learn general language representations that transfer across domains, reducing the amount of domain-specific training data needed while maintaining high classification accuracy through efficient few-shot learning.
Solution Approach 2:
The patent changes the approach from requiring extensive domain-specific data to using parameter-efficient adaptation methods. By freezing most pre-trained parameters and only training adaptation layers, the system achieves high accuracy with minimal additional training data, effectively changing the data efficiency parameters of the training process.
Data Source
AI summary
Embodiments described herein provide a cross-lingual intent classification model that predicts in multiple languages without the need of training data in all the multiple languages. For example, data requirement for training can be reduced to just one utterance per intent label. Specifically, when an utterance is fed to the intent classification model, the model checks whether the utterance is similar to any of the example utterances provided for each intent. If any such utterance(s) are found, the model returns the specified intent, otherwise, it returns out of domain (OOD).