Iterative Term Extraction for Domain Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing natural language processing (NLP) technologies face challenges in term extraction when dealing with texts from different domains than those they were built for, often resulting in low precision and recall, and inability to discover new domain-specific terms effectively.
Innovation Solution
The method iteratively extracts contextually relevant terms by using weighted contextual characteristics of previously discovered terms, adding them to a high precision domain lexicon, and refining the list of candidate terms through statistical and association rule-based filtering, without relying on linguistic processors or dictionaries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If existing NLP technologies are used for term extraction in a different domain, then the system can process texts from that domain, but the precision and recall of extracted terms deteriorate
Solution Approach 1:
The system automatically learns domain-specific terms and contextual characteristics from the corpus itself through iterative extraction, rather than relying on pre-built domain ontologies or external linguistic resources. The discovered terms are added back to expand the lexicon for subsequent iterations, enabling the system to self-adapt to new domains
Solution Approach 2:
The system dynamically adjusts the lexicon and contextual characteristics based on discovered terms from each iteration. By changing the parameter set (lexicon + contextual characteristics) iteratively, the system adapts to domain-specific language patterns while maintaining high extraction precision
2Adaptability or versatility
If existing NLP technologies are used for term extraction in a different domain, then the system can process texts from that domain, but the recall of extracted terms deteriorates
Solution Approach 1:
The system continuously expands its term lexicon by adding newly discovered terms from each iteration, enabling it to progressively improve recall for domain-specific terms that were not present in the initial lexicon or training data
Solution Approach 2:
The system uses the contextual characteristics of discovered terms to guide subsequent term extraction iterations. This feedback loop allows the system to progressively discover more domain-specific terms, improving recall over time
3Adaptability or versatility
If iterative term extraction with lexicon expansion is performed, then term discovery capability improves, but computational resources and processing time increase
Solution Approach 1:
The system maintains continuous improvement of term discovery capability through iterative extraction, where each iteration builds upon the previous one by adding discovered terms to the lexicon, ensuring that the useful action of term discovery continues without interruption
Solution Approach 2:
The system performs partial iterations of the extraction process, allowing users to stop at any point when sufficient terms are discovered. This enables balancing between term discovery capability and processing time based on specific needs
Data Source
AI summary
Software that extracts contextually relevant terms from a text sample (or corpus) by performing the following steps: (i) identifying a first term from a corpus, based, at least in part, on a set of initial contextual characteristic(s), where each initial contextual characteristic of the set of initial contextual characteristic(s) relates to the contextual use of at least one category related term of a set of category related term(s) in the corpus; (ii) adding the first term to the set of category related term(s), thereby creating a revised set of category related term(s) and a set of first term contextual characteristic(s), where each first term contextual characteristic of the set of first term contextual characteristic(s) relates to the contextual use of the first term in the corpus; and (iii) identifying a second term from the corpus, based, at least in part, on the set of first term contextual characteristic(s).