Lexical Dropout Training for Context-Aware Entity Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial intelligence-based chatbots struggle to accurately identify named entities due to insufficient attention on contextual information during training, leading to misclassification of entities.
Innovation Solution
Implementing lexical dropout techniques to selectively drop out feature vectors associated with entities during training, focusing the model on contextual information using a dropout parameter and hypertuning to determine the appropriate dropout rate.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the model is trained to focus on entity feature vectors, then entity recognition capability is improved, but contextual understanding deteriorates leading to misclassification
Solution Approach 1:
The patent extracts and removes entity feature vectors from the training process through selective dropout. By taking out the entity-specific feature vectors and preventing the model from learning their patterns, the method forces the model to rely on contextual information instead, thereby resolving the contradiction between entity recognition and contextual understanding.
Solution Approach 2:
The patent introduces dropout as an intermediary mechanism that mediates between entity features and contextual information. During training, dropout randomly masks entity feature vectors, acting as a mediator that prevents the model from over-relying on entities while still allowing contextual information to flow through and be learned.
2Loss of information
If dropout is applied to entity feature vectors during training, then contextual information processing is improved, but training complexity increases
Solution Approach 1:
The patent applies dropout selectively and locally only to entity feature vectors rather than uniformly across all features. This localized application of dropout to specific entity tokens allows the model to retain contextual information while avoiding unnecessary complexity in processing non-entity features, thus improving contextual retention without excessive training complexity.
3Measurement precision
If selective dropout is implemented for entity tokens, then named entity recognition accuracy is improved, but computational overhead during training increases
Solution Approach 1:
The patent implements partial dropout by applying the dropout mechanism only to entity feature vectors during training, rather than to all features. This partial action approach achieves improved named entity recognition accuracy by forcing contextual learning, while avoiding the excessive computational overhead of applying dropout uniformly across the entire feature space.
Data Source
AI summary
Techniques are provided for improved training of a machine learning model using lexical dropout. A machine learning model and a training data set are accessed. The training data set can include sample utterances and corresponding labels. A dropout parameter is identified. The dropout parameter can indicate a likelihood for dropping out one or more feature vectors for tokens associated with respective entities during training of the machine learning model. The dropout parameter is applied to feature vectors for tokens associated with respective entities. The machine learning model is trained using the training data set and the dropout parameter to generate a trained machine learning model. The use of the trained the machine learning model is facilitated.