Semantic Group Merging for Cross-Domain Prediction Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional AI-based models for chatbots are prone to inaccurate predictions due to limited training data in specific content domains, leading to irrelevant solutions that can adversely affect system operations.
Innovation Solution
A computer architecture that merges training data from multiple content domains, using pre-trained models like SBERT to identify similar groups and fine-tune vector representations, enhancing prediction accuracy by leveraging context across domains.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If training data is limited to a specific content domain, then the model maintains domain-specific accuracy, but the prediction accuracy deteriorates when training data is insufficient
Solution Approach 1:
The patent merges training data from multiple content domains by identifying and combining similar groups across domains. The system uses vector representations to find semantically similar groups from different domains and merges them into unified training sets, thereby increasing the quantity of training data while maintaining relevance through similarity-based selection.
Solution Approach 2:
The patent introduces vector representations as an intermediary mechanism to bridge different content domains. By converting text data into vector representations and comparing similarity metrics, the system can identify and merge groups from different domains that are semantically related, enabling cross-domain knowledge transfer without direct domain mixing.
2Quantity of substance
If training data from multiple content domains is used, then the quantity of training data increases, but the prediction accuracy deteriorates due to domain mismatch
Solution Approach 1:
The patent applies local quality by selectively merging only those groups from different domains that exhibit semantic similarity. Rather than uniformly combining all cross-domain data, the system identifies specific local regions (groups) within the data space that are similar across domains and merges only those, ensuring domain relevance while expanding training data quantity.
Solution Approach 2:
The patent changes the parameter of data selection from domain-based to similarity-based. By using vector representation similarity metrics as the selection criterion, the system transforms how training data is chosen and merged, allowing flexible combination of data from different domains based on semantic content rather than domain boundaries.
3Device complexity
If conventional AI-based models are used with limited domain-specific data, then the model architecture remains simple, but the prediction accuracy deteriorates
Solution Approach 1:
The patent performs preliminary action by pre-processing and organizing training data into vector representations and identifying similar groups across domains before model training. This preliminary organization of merged training data enables conventional models to achieve better accuracy without requiring complex architecture modifications.
Data Source
AI summary
Methods and systems are described herein for merging datasets from multiple content domains for training a prediction model to predict a solution for a request related to a specific content domain. A dataset for a content domain may include requests and solutions organized as groups. For example, a dataset for a first content domain may include a first group having (a) a first set of requests (e.g., questions or queries) related to a first topic, and (h) a solution (e.g., an answer) associated with the first set of requests. The datasets of different content domains are analyzed based on context-based vector representations of the requests or solutions to determine the groups that are similar and merge those similar groups into a single merged group. A prediction model is trained with the merged groups for obtaining a prediction of a solution to any given request.