Intent Classifier Training via Multi-Task BERT Fine-Tuning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current chatbots, both rule-based and AI-driven, face limitations in intent classification, particularly when dealing with questions that have multiple intents or are phrased grammatically incorrectly, leading to reduced performance and inability to handle dynamic language variations.
Innovation Solution
The development of a method and system for training an intent classifier using a fine-tuned Bidirectional Encoder Representations from Transformers (BERT) model, specifically through masked language modeling and multi-task deep neural networks, to generate domain-specific models that can better understand and classify intents in various industries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If rule-based chatbots use predefined questions and keywords, then consistency is improved, but adaptability deteriorates
Solution Approach 1:
The patent replaces the mechanical rule-based system with a neural network-based AI system. The chatbot uses natural language processing and machine learning models to understand and respond to user queries, eliminating the need for predefined rules and keyword matching while maintaining consistent performance through trained models.
Solution Approach 2:
The patent transforms the static parameters of rule-based chatbots into dynamic parameters through continuous learning. The system adjusts its response strategies based on learned patterns from training data, allowing it to adapt to new query types and language variations while maintaining performance consistency through the trained model architecture.
2Adaptability or versatility
If AI chatbots are trained extensively, then adaptability is improved, but training time deteriorates
Solution Approach 1:
The patent applies pre-training on large general language corpora before domain-specific fine-tuning. This preliminary action allows the model to learn fundamental language understanding and generation capabilities in advance, reducing the time required for subsequent domain-specific training while maintaining high adaptability to different domains.
Solution Approach 2:
The patent divides the training process into distinct stages: pre-training on general data, domain-specific fine-tuning, and task-specific adaptation. This segmentation allows each stage to focus on specific learning objectives, improving overall training efficiency while achieving comprehensive adaptability across different domains and tasks.
3Device complexity
If each question is mapped to one intent, then classification simplicity is improved, but measurement precision deteriorates
Solution Approach 1:
The patent implements dynamic intent classification where a single query can be mapped to multiple intents simultaneously. The neural network model outputs probability distributions across multiple intent classes, allowing flexible classification that adapts to the complexity of each query while maintaining a simple underlying architecture.
Data Source
AI summary
Methods and systems for training an intent classifier. For example, a question-intent tuple dataset comprising data samples is received. Each data sample has a question, an intent, and a task. A pre-trained language model is also received and fine-tuned by adjusting values of learnable parameters. Parameter adjustment is performed by generating a plurality of neural network models. Each neural network model is trained to predict at least one intent of the respective question having a same task value of the tasks of the question-intent tuple dataset. Each task represents a source of the question and the respective intent. The fine-tuned language model generates embeddings for training input data, the training input data comprising a plurality of data samples having questions and intents. Further, feature vectors for the data samples of the training input data are generated and used to train an intent classification model for predicting intents.