Health Data Indexing via Inverted Index and Segmented De-identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Healthcare entities face challenges in efficiently searching and analyzing aggregated patient data due to varying formats, levels of detail, and regulatory constraints, particularly with conventional de-identification techniques that often remove too much information, limiting data utility and being unsuited for handling data from different health systems.
Innovation Solution
A system and method for indexing and searching health data that aggregates data from multiple sources into a standard format, using a single user query to handle complex search requests through an inverted index mechanism, allowing efficient retrieval of patient records by constructing index terms and solving constraints, while ensuring data privacy and compliance with regulations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional de-identification techniques are used to protect patient privacy, then data privacy is improved, but data utility deteriorates due to excessive information removal
Solution Approach 1:
The patent segments the de-identification process into multiple stages: initial de-identification, data aggregation from multiple sources, and selective re-identification only when query constraints are satisfied. This segmented approach allows privacy protection while preserving data utility for research purposes.
Solution Approach 2:
The system dynamically changes the de-identification parameters based on query constraints and data sensitivity. Instead of applying fixed de-identification rules, the system adjusts the level of anonymization according to the specific research needs and compliance requirements, balancing privacy and utility.
2Quantity of substance
If data from multiple health systems is aggregated, then data quantity and diversity are improved, but data format consistency deteriorates
Solution Approach 1:
The patent introduces an intermediary normalization layer that sits between diverse health system data sources and the research query interface. This intermediary component translates various data formats into a unified structure without requiring changes to the source systems, enabling aggregation while maintaining format flexibility.
Solution Approach 2:
The system implements a universal data model that can accommodate multiple data formats and structures from different health systems. The normalized data structure serves multiple functions: preserving source data fidelity, enabling efficient querying, and supporting various research analyses simultaneously.
3Loss of information
If large volumes of patient data are stored for research, then research value is improved, but searching efficiency deteriorates
Solution Approach 1:
The patent applies preliminary indexing and normalization to patient data before it is stored in the aggregated database. By pre-processing the data into a standardized format with appropriate indexes, the system enables efficient querying of large datasets without sacrificing research completeness.
Solution Approach 2:
The system replaces traditional mechanical full-table scan searching with an inverted index mechanism that allows direct access to relevant patient records based on query constraints. This substitution dramatically reduces search time while maintaining the ability to analyze large volumes of patient data.
Data Source
AI summary
Systems and methods for indexing and searching health data are disclosed herein. In some embodiments, a method for querying patient records includes receiving a search input that specifies a plurality of events, one or more temporal relationships between the events, and inclusion and/or exclusion criteria. An index query is constructed based at least in part on the plurality of events within the search input. The index query is then executed against an inverted index to identify matching patient records. One or more temporal constraints can be solved against the identified patient records. Finally, patient records satisfying the constraint(s) are returned.