FHIR Data Loading via Threaded Queues and Sequential Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in ensuring that healthcare data is loaded into FHIR servers in a concurrent environment while maintaining point-in-time architecture and chronological order, and eliminating sequential repeats to prevent version bloat and reduce communication between the FHIR loader and server.
Innovation Solution
A computing system is designed to receive data files, align rows by unique IDs, compress rows with identical information, and load the processed data into a FHIR server using multiple threads to maintain chronological order and eliminate sequential repeats.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data files are loaded into FHIR server using multiple threads in concurrent environment, then loading efficiency is improved, but maintaining chronological order and point-in-time architecture becomes difficult
Solution Approach 1:
The patent segments the data loading process by creating separate queues for each thread, where each queue maintains its own chronological order. Threads process rows independently within their assigned queues, preventing interleaving operations that would disrupt chronological ordering. This segmentation allows concurrent processing while preserving the required temporal sequence for each resource type.
Solution Approach 2:
The patent introduces an intermediary mechanism (the threaded queue system) that mediates between the concurrent data sources and the FHIR server. Each thread acts as an intermediary that processes rows in chronological order before submitting to the server, ensuring that point-in-time architecture requirements are met even during concurrent operations.
2Quantity of substance
If all rows are loaded into FHIR server without compression, then data completeness is maintained, but version bloat occurs and communication overhead increases
Solution Approach 1:
The patent applies partial action by selectively compressing only those rows that represent actual data changes, rather than processing every row. The system identifies and compresses sequential repeats (rows with unchanged data from the previous version), eliminating unnecessary version creations while preserving data completeness for actual changes.
Solution Approach 2:
The patent changes the parameter of data representation by introducing a compression state. Rows are transformed from full detailed records to compressed representations when they contain unchanged data, reducing the quantity of data transmitted to and stored in the FHIR server while maintaining the ability to reconstruct complete data when needed.
3Loss of energy
If data compression is applied to eliminate sequential repeats, then communication overhead is reduced, but processing complexity increases
Solution Approach 1:
The patent applies preliminary action by performing data compression and comparison operations before data transmission to the FHIR server. Rows are pre-processed to identify sequential repeats and compress unchanged data, reducing the volume of communication required. This preliminary processing shifts complexity from the communication phase to the preparation phase, where it can be more efficiently managed.
Data Source
AI summary
A data file comprising a plurality of rows, each of the rows includes at least a first column and a second column, the first column contains a first-level resource ID that identifies a first-level resource, the second column provides information regarding the first-level resource. For each respective unique first-level resource ID, the processing circuitry identifies a row set for the respective first-level resource ID, performs a sequential deduplication process on the row set, and enqueues remaining rows in a queue for a thread assigned to the respective first-level resource ID. For each row enqueued in the queue for the thread, the thread dequeues the row from the queue for the respective thread, requests creation of a second-level resource that stores a version of the data element contained in the second column of the dequeued row, and requests creation of relationship data for the second-level resource.