Forced-Prompting Data Normalization for Free-Text Patient Records

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Legacy databases like MUMPS store patient information in free-text format, leading to inconsistent data type representations, which complicates data aggregation and requires significant manual intervention for cleaning and normalization.

Innovation Solution

Utilize machine learning systems, particularly large language models, to force hallucinations and identify correlations between disparate data types within datasets, enabling automatic normalization and unification of data types with reduced human intervention.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If machine learning models are used to automatically normalize data types, then productivity and automation extent are improved, but measurement precision and reliability may deteriorate due to potential hallucinations and incorrect correlations

Engineering Contradiction:

Improvedata normalization speedVSAvoiddata type identification accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The system implements an iterative feedback loop where the ML model generates initial data type correlations, human reviewers provide feedback on accuracy, and the model is retrained using the corrected data. This continuous feedback mechanism allows the system to maintain high productivity while progressively improving measurement precision through human-in-the-loop validation.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system uses partial action by applying ML models only to high-volume, low-risk data normalization tasks while reserving manual review for critical data types. The model generates excessive initial correlations (more than strictly necessary) to ensure comprehensive coverage, then filters and validates the most relevant correlations through feedback mechanisms, balancing automation extent with accuracy requirements.

Inventive Principle:

Principle #16Partial or excessive action

2Measurement precision

If manual review and cleaning processes are used, then measurement precision is improved, but loss of time and productivity deteriorate

Engineering Contradiction:

Improvedata type categorization accuracyVSAvoidmanual cleaning time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system performs preliminary action by using ML models to pre-process and correlate the majority of data entries before human review. The model generates initial data type assignments and correlations in advance, so that when human reviewers examine the data, only the most challenging and ambiguous cases require manual intervention, significantly reducing overall time loss while maintaining precision.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The data cleaning process is segmented into distinct phases: automated ML-based correlation identification, intermediate validation layer, and final human review for edge cases. This segmentation allows each phase to focus on specific tasks, with the automated phase handling volume and the human phase ensuring precision, thereby optimizing the balance between time efficiency and accuracy.

Inventive Principle:

Principle #1Segmentation

3Ease of operation

If free-text format storage is used, then ease of operation and data entry flexibility are improved, but device complexity and data aggregation difficulty worsen

Engineering Contradiction:

Improvedata entry flexibilityVSAvoiddata aggregation complexity

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The ML correlation model acts as an intermediary layer between the free-text storage system and the structured data aggregation system. It translates diverse free-text representations into standardized data type correlations, mediating between the simplicity of free-text entry and the complexity of structured aggregation, thereby maintaining ease of operation while enabling efficient data consolidation.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system changes the parameter representation by transforming free-text data into structured correlation parameters that capture the essential meaning of each data type. Instead of working with raw text strings, the system uses standardized parameters (data types, units, formats) that are derived from the free-text through ML correlation analysis, simplifying aggregation operations while preserving the flexibility of free-text input.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS20260073226A1Systems and methods for data normalization using forced prompting with machine learning models

Publication Date: 2026.03.12 NORTHWESTERN MEMORIAL HEALTHCARE

AI summary

Systems and methods include techniques associated with one or more machine learning systems to normalize disparate entries within one or more datasets for common data types. The one or more machine learning systems may be used to generated relationships between attribute-value pairs associated with a particular data type and then to determine, from a corpus of free-form data, individual entries for a target data type. The identified individual entries may be used to extract information from the dataset and generate a modified, clean dataset.