Limited Data Enricher Using Transfer Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Healthcare data is typically collected for specific purposes, leading to limited feature sets, making it costly and inefficient to add new features, as it requires updates to data collection protocols, training, and infrastructure, without ensuring the benefits outweigh the costs.
Innovation Solution
A method that utilizes richer datasets to enrich limited datasets by determining feature estimators, updating them based on additional features, and providing recommendations for data collection to improve analytical outputs and target insights, using a system that identifies population relatedness and applies transfer learning methods.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If additional features are collected to enrich the dataset, then the accuracy and completeness of data analysis is improved, but the cost and complexity of data collection and processing increases
Solution Approach 1:
The patent introduces an intermediary approach by using a rich dataset as a mediator to enhance the limited dataset. Instead of directly collecting all desired features through complex data collection processes, the system uses transfer learning to map features from the rich dataset to the limited dataset, thereby improving analysis accuracy without proportionally increasing collection complexity
Solution Approach 2:
The patent applies copying by creating a simplified representation of the rich dataset's features through transfer learning. The system copies relevant feature relationships and patterns from the comprehensive rich dataset and adapts them to the limited dataset, allowing the limited dataset to gain insights typically requiring much more extensive data collection
2Quantity of substance
If new data collection protocols and training are implemented to gather additional features, then the feature set is enriched, but the time and resources required for implementation increases
Solution Approach 1:
The patent applies preliminary action by pre-processing and preparing the rich dataset in advance. The system pre-extracts feature relationships, patterns, and insights from the rich dataset before they are needed for the limited dataset analysis. This allows the limited dataset to benefit from pre-computed feature mappings and reduces the time required for implementation when working with the limited dataset
3Adaptability or versatility
If existing data collection infrastructure is expanded to include additional features, then the data comprehensiveness is improved, but the cost of infrastructure updates and maintenance increases
Solution Approach 1:
The patent applies universality by creating a multi-functional system where the transfer learning framework can handle multiple data collection scenarios. The same infrastructure can serve both the limited dataset and the rich dataset, and the system can adapt to different feature sets without requiring separate specialized infrastructure for each data collection effort, thereby reducing overall infrastructure update and maintenance costs
Data Source
AI summary
Embodiments of the present invention disclose a method, a computer program product, and a computer system for enriching data. A computer receives a limited dataset and a target insight. In addition, the computer identifies an applicable richer dataset and determines a population relatedness between the limited dataset and the applicable richer dataset. Moreover, the computer calculates estimators of features using the richer dataset as well as calculates estimators for features using the limited dataset. The computer then updates the estimators of the limited dataset using estimators of the richer dataset and evaluates the updated estimators. Lastly, the computer provides a data collection recommendation as it relates to the limited dataset based on the evaluation.