NLP Verbatim Narrative Mapping to Medical Terminology
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Clinical trials face challenges in consistently analyzing adverse events and concomitant medications due to the lack of standardization in verbatim narratives, leading to inconsistent mapping and classification using existing dictionaries like MedDRA and WHODD.
Innovation Solution
A system and method utilizing natural language processing and probabilistic models to preprocess and map verbatim narratives to standardized terms in medical and drug terminology dictionaries, including operations like cleaning, sorting, stemming, and applying synonyms or deletions to improve matching accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual mapping of verbatim narratives to standardized terms is performed, then mapping accuracy can be maintained, but processing time and labor costs increase significantly
Solution Approach 1:
The patent segments the mapping process into multiple independent stages: preprocessing (cleaning, lowercasing, removing punctuation), stemming, synonym expansion, and sequential matching against different dictionary levels (WHODD active ingredients, WHODD proprietary names, MedDRA preferred terms, MedDRA lowest level terms). This segmentation allows automated processing at each stage while maintaining overall mapping accuracy through progressive refinement.
Solution Approach 2:
The patent performs preliminary actions on verbatim narratives before mapping, including cleaning (removing punctuation, extra spaces), lowercasing, and stemming (reducing words to root forms). These preliminary transformations standardize the input data, enabling more accurate and efficient automated matching against terminology dictionaries without requiring manual intervention.
2Productivity
If automated mapping systems are implemented, then processing speed increases, but mapping accuracy and consistency deteriorate due to variability in verbatim narratives
Solution Approach 1:
The patent changes multiple parameters of the verbatim narratives to improve automated matching: converting to lowercase, removing punctuation marks, eliminating extra whitespace, and applying stemming algorithms. These parameter transformations normalize the variable input data, enabling consistent automated processing while maintaining mapping accuracy despite the inherent variability in narrative formats.
Solution Approach 2:
The patent introduces intermediate processing steps between the raw verbatim narrative and the final standardized term mapping. These intermediaries include preprocessing functions (cleaning, lowercasing), stemming operations, and synonym expansion. Each intermediary transforms the data progressively, bridging the gap between variable narrative formats and standardized terminology while maintaining accuracy through controlled transformations.
3Measurement precision
If comprehensive preprocessing and multiple matching stages are applied, then mapping coverage and accuracy improve, but system complexity increases
Solution Approach 1:
The patent divides the complex mapping system into segmented, modular stages: preprocessing module, stemming module, synonym expansion module, and four sequential matching modules (WHODD active ingredients, WHODD proprietary names, MedDRA preferred terms, MedDRA lowest level terms). Each module performs a specific function and can be independently maintained, reducing overall system complexity while achieving comprehensive mapping coverage through the combination of all modules.
4Stability of the object's composition
If existing dictionary-based mapping solutions are used, then standardization is achieved, but consistency in mapping results deteriorates due to variations in verbatim narrative formatting
Solution Approach 1:
The patent applies systematic parameter changes to all verbatim narratives before dictionary matching: converting to lowercase, removing all punctuation marks, eliminating leading/trailing and excessive internal whitespace. These consistent transformations eliminate formatting variations that would otherwise cause inconsistent mapping results, ensuring that the same narrative content always maps to the same standardized term regardless of original formatting.
Data Source
AI summary
An apparatus for automatically mapping a verbatim narrative to a term in a medical terminology dictionary includes a natural language processor and a comparator. The natural language processor processes terms from the medical terminology dictionary and from a medical coding decision database to generate a processed database that also includes the original terms from the medical terminology dictionary and the medical coding decision database. The natural language processor also processes the verbatim narrative. The comparator compares the processed verbatim narrative to the terms in the processed database and determines whether the processed verbatim narrative is an exact match to a term in the processed database. The verbatim narrative is mapped to the term in the medical terminology dictionary that corresponds to the term in the processed database that is an exact match. The verbatim narratives may include adverse event narratives, concomitant medication narratives, or other types of narratives. A method for automatically mapping a verbatim narrative to a term in a medical terminology dictionary is also described and claimed.