Automated Clinical Annotation via EHR Data Linking

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The current manual process for generating ground truth annotations for clinical text in electronic health records is time-consuming, requires trained experts, and is prone to human error, making it inefficient and costly for training machine learning models.

Innovation Solution

Automating the generation of 'silver standard' ground truth by linking unstructured clinical notes to structured data entries, using derived insights and metadata to define criteria for annotation, and applying natural language processing techniques for filtering and quality improvement.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If manual annotation by medical experts is used to generate ground truth, then annotation quality and reliability are improved, but time consumption and cost increase significantly

Engineering Contradiction:

Improveannotation qualityVSAvoidtime consumption

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent uses structured EHR data as a copy or proxy representation of clinical information to automatically generate annotations. Instead of requiring experts to manually annotate unstructured clinical notes, the system copies relevant information from structured data fields (such as diagnosis codes, medication lists, vital signs) to automatically create ground truth labels, thereby eliminating time-consuming manual annotation while maintaining annotation quality

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The system enables self-service annotation by automatically generating ground truth labels using the EHR data itself. The structured data within the EHR serves as the source for automatic annotation generation, allowing the data to annotate itself without human intervention. This self-service approach maintains reliability by using the same data source that contains the clinical information, while dramatically reducing time consumption

Inventive Principle:

Principle #25Self-service

2Measurement precision

If manual annotation by medical experts is used to generate ground truth, then annotation accuracy is improved, but the process becomes expensive and requires specialized resources

Engineering Contradiction:

Improveannotation accuracyVSAvoidprocess cost

Core Design Contradiction:

Measurement precisionVSEase of manufacture

Solution Approach 1:

The patent replaces expensive manual annotation processes with automated copying of information from structured EHR data fields. The system copies relevant clinical information from structured data (such as ICD-10 diagnosis codes, CPT procedure codes, medication lists) to automatically generate accurate annotations, eliminating the need for paid medical expert annotators while maintaining high annotation accuracy

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The system substitutes the mechanical process of manual expert annotation with an automated computational process. Instead of human experts manually reviewing and labeling clinical notes, the system uses algorithmic processing to automatically extract and label information from structured data fields, replacing the expensive human resource requirement with efficient automated processing

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Productivity

If automated annotation methods are used to reduce time consumption, then productivity is improved, but annotation quality and reliability may deteriorate

Engineering Contradiction:

Improveannotation speedVSAvoidannotation quality

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent introduces structured EHR data as an intermediary between the automated system and the unstructured clinical notes. The structured data serves as a mediator that contains pre-organized, machine-readable clinical information which the automated system can reliably extract and use for annotation. This intermediary approach ensures that automated processing maintains high reliability by working with data that is already in a standardized, trustworthy format

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system performs preliminary organization of clinical information into structured data formats before automated annotation occurs. By pre-structuring the EHR data into standardized fields (diagnosis codes, medication lists, vital signs), the system prepares the data in advance for reliable automated processing. This preliminary action ensures that when automated annotation runs, it can quickly and accurately extract information without compromising quality

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11782942B2Auto-generating ground truth on clinical text by leveraging structured electronic health record data

Publication Date: 2023.10.10 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

A method improves performance of natural language processing by automatically generating ground truth from electronic health records comprising unstructured clinical notes and structured data comprising entries each having respective values for fields. The method includes: linking a given one of the notes to a given one of the entries responsive to determining that a specified field within the given entry matches an item of metadata for the given note; determining an initial set of the notes which satisfy criteria selected such that the criteria are a proxy for the ground truth, wherein the given note is determined to satisfy the criteria based at least in part on the given entry linked thereto; and designating at least a portion of the initial set of notes which satisfy the criteria, and the entries linked to the portion of the initial set of notes which satisfy the criteria, as the ground truth.