Cross-Domain NER Prompting With Similarity-Selected Examples
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Named Entity Recognition (NER) systems require large volumes of labeled data for training and are often domain-specific, making them costly and time-intensive, and they struggle to adapt effectively to data-scarce target domains.
Innovation Solution
A system using a large language model (LLM) with a dynamically generated input prompt based on computed similarity with the textual data, allowing domain adaptation without adjusting model parameters, and leveraging labeled examples from data-rich source domains for NER in data-scarce target domains.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional NER systems use large volumes of labeled data for training, then entity recognition accuracy is improved, but training time and computational resources increase significantly
Solution Approach 1:
The system performs preliminary action by using the LLM to generate synthetic labeled training data before the actual NER training process. This pre-generated synthetic data serves as high-quality training material that would otherwise require extensive manual annotation, thereby reducing the time needed for data preparation and model training while maintaining accuracy.
Solution Approach 2:
The system creates copies of existing labeled data from source domains by using the LLM to generate synthetic training examples that replicate the structure and quality of real labeled data. These synthetic copies can be used to train NER models without requiring collection and annotation of equivalent volumes of new data, significantly reducing training time and resources.
2Measurement precision
If NER systems are trained on domain-specific data, then recognition accuracy for that domain is improved, but adaptability to new domains deteriorates
Solution Approach 1:
The system achieves universality by training the NER model on synthetic data generated from multiple source domains using the LLM. This multi-domain synthetic training data enables the model to learn domain-agnostic entity patterns while maintaining the ability to adapt to specific target domains, thus achieving both domain-specific accuracy and cross-domain versatility.
Solution Approach 2:
The system applies parameter changes by dynamically adjusting the domain-specific parameters and entity type definitions in the synthetic data generation process. The LLM generates synthetic training data with configurable domain parameters, allowing the same base model to adapt to different target domains by changing the input parameters rather than retraining the entire model architecture.
3Adaptability or versatility
If multiple domain-specific NER models are maintained, then coverage across domains is improved, but model maintenance complexity and costs increase
Solution Approach 1:
The system implements a single universal NER model that can handle multiple domains through synthetic data generation. Instead of maintaining separate domain-specific models, the universal model is trained on synthetic data from various domains and can adapt to new domains as needed, significantly reducing model maintenance complexity while preserving broad domain coverage.
Solution Approach 2:
The system creates synthetic copies of domain-specific training data on-demand using the LLM, eliminating the need to maintain multiple actual domain-specific models. When a new domain is needed, the system generates synthetic training data for that domain and fine-tunes the universal model, rather than maintaining a separate pre-trained model for each domain.
4Measurement precision
If labeled data is collected from data-scarce target domains, then NER accuracy for target domain is improved, but data collection cost and time increase
Solution Approach 1:
The system creates synthetic copies of labeled training data for the target domain using the LLM, based on available unlabeled data or patterns from source domains. This synthetic data copying approach provides sufficient training material to achieve good NER accuracy in the target domain without requiring collection and annotation of large volumes of actual target domain data.
Solution Approach 2:
The system performs preliminary action by generating synthetic labeled data for the target domain before actual NER training begins. This pre-generated synthetic data serves as a substitute for collecting and annotating real target domain data, reducing both the time and resources required for data collection while providing adequate training material for accurate NER performance.
Data Source
AI summary
Systems and methods for automated named entity recognition (NER) using artificial intelligence models are disclosed. In some examples, a contextualized word embedding is generated for each of a plurality of words. Further, for each contextualized word embedding, example contextualized word embeddings are received. Each of the example contextualized word embeddings are associated with a corresponding digital textual example. A similarity value is generated between each contextualized word embedding and each of the corresponding example contextualized word embeddings. Based on the similarity values, one or more of the contextualized word embeddings are determined. An input prompt is generated that includes a command, the plurality of words, and the digital textual example associated with each of the determined contextualized word embeddings. The input prompt is then inputted to a generative artificial intelligence model to receive a response that associates at least one of the plurality of words with an entity type.