Transformer Embeddings for Hierarchical Claims Data Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
There is a lack of a publicly available foundational machine learning model trained on a large repository of claims data to generate predictions such as future claims-related insights, cost, utilization, disease progression, patient similarity, and clinical insights effectively.
Innovation Solution
A transformer-based model system that generates predictions by using multiple machine learning models to create base, event, and time embeddings, allowing for the generation of aggregated embeddings for specified entities, which are then used to make predictions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple machine learning models are used to generate different types of embeddings (base, event, time), then prediction accuracy is improved, but device complexity increases
Solution Approach 1:
The patent segments the embedding generation process into three distinct machine learning models, each responsible for a specific type of embedding (base embeddings for code semantics, event embeddings for historical events, and time embeddings for temporal patterns). This segmentation allows each model to specialize in capturing particular aspects of the data, improving overall prediction accuracy while maintaining manageable complexity through modular design.
Solution Approach 2:
The multiple embedding models serve universal functions by collectively representing different dimensions of the same entity (patient). Rather than requiring separate systems for different prediction tasks, the plurality of models work together to provide comprehensive representations that can be applied across various prediction scenarios, reducing overall system complexity while maintaining high accuracy.
2Measurement precision
If large volumes of claims data are processed to train the models, then prediction accuracy is improved, but loss of time increases
Solution Approach 1:
The patent applies preliminary action by pre-training the multiple embedding models on large volumes of historical claims data before deployment. The base embedding model learns code semantics, the event embedding model learns historical event patterns, and the time embedding model learns temporal relationships in advance. This pre-training allows the system to quickly generate accurate predictions for new entities without requiring real-time processing of large datasets, thus reducing inference time while maintaining high accuracy.
3Loss of information
If embeddings are generated for multiple dimensions (semantic, event-based, temporal), then information completeness is improved, but device complexity increases
Solution Approach 1:
The patent employs dimensionality change by representing multi-dimensional information (semantic, event-based, and temporal characteristics) as vector embeddings in a unified mathematical space. Each type of information is transformed into a numerical vector representation, allowing the system to capture complex relationships across multiple dimensions while simplifying subsequent processing through standard vector operations and distance measurements.
Data Source
AI summary
A method includes generating a base code embedding in a first vector space for a specified entity using at least one first machine learning model based on a set of codes corresponding to the entity from a claims datastore. The method includes generating an event embedding in a second vector space for the specified entity using a second machine learning model based on a set of historical events corresponding to the specified entity and the base code embedding, and generating a time embedding in a third vector space for the specified entity using a third machine learning model based on times between consecutive ones of the set of historical events and the base code embedding. The method includes generating an aggregated embedding for the specified entity based on the event embedding and the time embedding, and generating a prediction by supplying the aggregated embedding to a fourth machine learning model.