Iterative Text Fragment Annotation for Medical Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The annotation of unstructured patient data for AI applications is slow, expensive, and prone to errors due to the manual and time-consuming process of classifying large volumes of health records, which are often written in varying terminologies by different healthcare providers.
Innovation Solution
A system and method for generating an annotated structured dataset from unstructured patient data using a search engine that iteratively presents subsets of unclassified text fragments to the user for classification, allowing for single-click selections and updates the search engine with user feedback to improve accuracy and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual annotation methods are used to classify health records, then annotators can review and label each record, but the process becomes extremely slow and time-consuming
Solution Approach 1:
The patent segments the annotation task by dividing unstructured text into smaller text fragments and presenting them in manageable batches of 50-200 fragments at a time. This allows annotators to work through large volumes of data efficiently while maintaining accuracy, as each batch is small enough to review carefully but the overall process remains fast due to automated batch processing
Solution Approach 2:
The patent introduces an automated search engine and text processing system as an intermediary between the raw unstructured data and the final annotated output. This intermediary automatically performs initial text fragmentation, searching, and filtering, reducing the manual workload while preserving annotation quality through human-in-the-loop verification
2Manufacturing precision
If annotators manually review each health record, then they can apply medical classification terms, but the annotation process becomes expensive and resource-intensive
Solution Approach 1:
The patent performs preliminary automated actions including text fragmentation, initial searching using medical classification terms, and filtering of relevant text fragments before presenting them to annotators. This preliminary processing reduces the volume of text requiring manual review while ensuring that only relevant fragments are annotated, improving both accuracy and throughput
Solution Approach 2:
The patent implements continuous automated processing that operates parallel to human annotation. While annotators work on one batch, the system continuously processes the next batch, searches for additional relevant fragments, and updates the annotation state. This continuous automated action maintains high productivity without compromising classification accuracy
3Quantity of substance
If manual annotation is performed on large volumes of unstructured data, then complete coverage is achieved, but errors increase due to the tedious nature of the task
Solution Approach 1:
The patent implements feedback mechanisms where annotator corrections and confirmations are used to refine the automated search and filtering process. The system learns from annotator decisions about which fragments are relevant, improving the accuracy of subsequent automated retrieval and reducing errors in future annotation batches while maintaining comprehensive data coverage
Data Source
AI summary
There is provided a system and a method of generating an annotated structured dataset, comprising: receiving a medical classification term, searching over the unstructured patient data for extracting unclassified unstructured text fragments, presenting a subset of the unclassified unstructured text fragments, receiving an indication of a selection of none or at least one of the text fragments, and one of: (i) classifying non-selected unclassified unstructured text fragments according to the medical classification term, and classifying selected text fragments as not satisfying the medical classification term, and (ii) classifying selected unclassified unstructured text fragments according to the medical classification term, and classifying non-selected unclassified unstructured text fragments as not satisfying the medical classification term, and iterating the searching, and/or the presenting, until no text fragments are obtained by the search, wherein the annotated structured dataset is created by the classification of unclassified unstructured text fragments into the medical classification term.