Closed Domain Question Answering Model Ensemble
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Question answering models trained on general purpose language data perform poorly in closed domains due to insufficient training data and de-emphasis on domain-specific concepts, leading to reduced accuracy in answering questions relevant to closed domains.
Innovation Solution
Implementing transfer learning to generate an ensemble of closed domain models by parsing training data into subsets based on out-of-vocabulary words and structural differences, and selecting appropriate models based on the number of OOV words and passage structure to improve performance in closed domains.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If question answering models are trained on vast amounts of general purpose language domain data, then the models achieve human level performance in general domains, but the training becomes cost and time prohibitive and the models do not work well in closed domains
Solution Approach 1:
The patent segments the general purpose language model into multiple specialized models, each trained on specific closed domain data subsets. This segmentation allows each model to focus on domain-specific concepts and terminology, achieving high accuracy in closed domains without requiring retraining on entire general purpose datasets, thus reducing training time while maintaining performance.
Solution Approach 2:
The patent applies local quality by creating models with specialized vocabulary and parameters tailored to specific closed domains. Each domain-specific model incorporates domain-specific word embeddings and attention mechanisms optimized for local domain characteristics, rather than using uniform general purpose parameters, thereby achieving high accuracy in target domains with reduced training requirements.
2Adaptability or versatility
If question answering models are trained on general purpose language data, then they achieve broad applicability, but they de-emphasize domain-specific concepts and perform poorly in closed domains
Solution Approach 1:
The patent divides the single general purpose model into multiple specialized models, each segment focused on a specific closed domain. This segmentation enables each model to specialize in domain-specific concepts and terminology, achieving high accuracy in closed domains while the collection of models maintains broad applicability across different domains.
Solution Approach 2:
The patent changes parameters by training each specialized model with domain-specific hyperparameters, vocabulary sizes, and embedding dimensions optimized for particular closed domains. This parameter customization allows models to capture domain-specific concepts effectively, improving closed domain accuracy while maintaining versatility through multiple specialized instances.
3Measurement precision
If an ensemble of specialized models is created for closed domains, then accuracy in closed domains improves significantly, but the system complexity increases
Solution Approach 1:
The patent segments the model selection process into discrete, manageable components: domain identification module, model selection module, and inference module. Each component handles a specific aspect of the ensemble system, making the overall complex system more manageable and interpretable while achieving high closed domain accuracy through specialized models.
4Measurement precision
If training data is parsed into subsets based on out-of-vocabulary words and structural differences, then model performance in closed domains improves, but the data processing complexity increases
Solution Approach 1:
The patent segments training data into distinct subsets based on out-of-vocabulary word thresholds and structural characteristics. This segmentation creates clearly defined data categories that can be efficiently processed and assigned to appropriate specialized models, improving closed domain accuracy while managing data processing complexity through systematic categorization.
Data Source
AI summary
Discussed herein are devices, systems, and methods for determining an answer to a natural language question. A method can include receiving a question and a passage to be used to answer the question and executing (i) a first trained ML model, based on the passage and the question and in response to determining that the question, passage, or a combination thereof includes more than a first threshold number of out of vocabulary (OOV) words, relative to a general purpose language ML model, to generate a first answer to the question or (ii) executing a second trained ML model, based on the passage and the question and in response to determining the question, passage, or a combination thereof includes less than the first threshold number of out of vocabulary (OOV) words, relative to the general purpose language ML model, to generate a second answer to the question.