Rare Disease Dictionary Augmentation Using Semantic Term Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Manual curation of disease dictionaries, particularly for rare diseases, is labor-intensive and suffers from low recall, limiting their effectiveness in Biomedical Natural Language Processing tasks.
Innovation Solution
A processor-implemented method and system that uses pattern-based and n-gram-based noun phrase chunking, semantic analysis, and filtering techniques to augment rare disease dictionaries by extracting relevant terms from medical literature, leveraging syntactic and semantic similarity measures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual curation is used to create disease dictionaries, then precision is improved, but recall deteriorates and time consumption increases
Solution Approach 1:
The patent segments the dictionary creation process into multiple stages: initial manual curation for high-quality seed terms, automated extraction of candidate terms from literature, semantic similarity-based filtering, and final manual review. This segmentation allows different methods to be applied at different stages, achieving both high precision and improved recall.
Solution Approach 2:
The patent introduces an intermediary automated processing layer between manual curation and final dictionary output. This layer uses NLP techniques to extract candidate terms and compute semantic similarities, acting as a mediator that expands coverage while maintaining quality through subsequent filtering steps.
2Measurement precision
If manual curation is used to create disease dictionaries, then precision is improved, but productivity deteriorates
Solution Approach 1:
The patent divides the time-consuming manual curation task into segments: initial manual creation of a small seed dictionary, automated expansion to generate candidate terms, and selective manual review of only the most promising candidates. This segmentation dramatically reduces total time while maintaining precision.
Solution Approach 2:
The patent performs preliminary automated processing to generate and pre-filter candidate terms before manual review. By pre-computing semantic similarities and filtering low-quality candidates, the system reduces the time burden on manual curators to reviewing only high-potential terms.
3Quantity of substance
If automated methods are used to extract terms from literature, then recall is improved, but precision deteriorates
Solution Approach 1:
The patent implements a feedback loop where extracted candidate terms are evaluated against semantic similarity thresholds and quality filters. Terms that meet predefined criteria are retained and fed into the next processing stage, while others are rejected. This feedback mechanism ensures precision is maintained even as recall increases through automated extraction.
Solution Approach 2:
The patent changes the parameter of term selection from simple keyword matching to semantic similarity-based selection. By computing semantic distances between candidate terms and seed terms, the system can identify truly relevant terms while filtering out spurious matches, thereby maintaining precision while improving recall.
Data Source
AI summary
Comprehensive and high-quality disease dictionaries are invaluable resources for tasks such as building ontologies, automated relation extraction, text summarization, question answering etc. Such curated resources are useful to clinicians, researchers, and various Biomedical Natural Language Processing tasks. However, these are manually curated and are labor and time intensive, and additionally suffer from lower recall and coverage is also less. Present disclosure provides systems and methods for augmenting rare disease dictionaries, wherein the system retrieves (new) rare diseases terms from medical literature that are related to the given dictionary terms (seed terms) and recommends new terms (or NPs) in a ranked order. This method is useful for rare diseases dictionary augmentation as a significant fraction of the top recommendations are new synonym candidates for dictionary augmentation. The method uses syntactic and semantic similarity measures in combination with efficient nearest neighbor search for efficient retrieval.