AI-Assisted NPI Data Curation System
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data curation initiatives for non-pharmaceutical intervention (NPI) data from heterogeneous sources are labor-intensive, time-consuming, and result in datasets with varying degrees of coverage, freshness, and granularity, leading to reduced accuracy in machine learning models predicting NPI strategies.
Innovation Solution
A system utilizing artificial intelligence (AI) to extract and curate NPI events from heterogeneous data sources using a machine learning framework that includes an extraction component for identifying candidate NPI events and a change detection component for evaluating their inclusion in a standardized dataset, employing natural language processing and a 5-tuple representation to facilitate robust and frequently updated NPI data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual data curation methods are used for NPI data from heterogeneous sources, then data can be curated with human oversight and validation, but the process becomes labor-intensive and time-consuming
Solution Approach 1:
The patent introduces an AI-based extraction component as an intermediary between heterogeneous data sources and the final curated dataset. This intermediary automatically extracts candidate NPI events from diverse sources (news articles, social media, government reports) and presents them for human validation, significantly reducing the manual curation workload while maintaining data quality through the defined format structure with fields for event type, location, date, and description
Solution Approach 2:
The system performs preliminary extraction and organization of NPI events from heterogeneous sources before human validation. The extraction component pre-processes raw data by identifying candidate events, categorizing them by type (e.g., travel restrictions, gathering limitations), and structuring them in a standardized format, so that human curators only need to review and validate rather than create entries from scratch
2Quantity of substance
If comprehensive data collection from multiple heterogeneous sources is performed, then dataset coverage is improved, but data consistency and formatting become more difficult to maintain
Solution Approach 1:
The patent implements a universal defined format structure that can accommodate NPI events from multiple heterogeneous data sources. The standardized schema includes flexible fields (event type, location, date, description, source) that can capture diverse information from different source types (news, social media, government reports) while maintaining consistent data organization, enabling the system to process various source formats through a single unified interface
Solution Approach 2:
The system transforms unstructured and semi-structured data from heterogeneous sources into a standardized parameterized format. By converting diverse source data into consistent fields (event type categorization, standardized location names, normalized date formats, structured descriptions), the system maintains data consistency across different sources while preserving comprehensive coverage of NPI events
3Loss of time
If frequent updates to NPI datasets are implemented to maintain freshness, then predictive model accuracy is improved, but computational resources and processing time increase
Solution Approach 1:
The system implements periodic updating of the NPI dataset by scheduling regular extraction and validation cycles. The extraction component operates at defined intervals to collect new NPI events from heterogeneous sources, update the curated dataset, and retrain predictive models periodically rather than continuously, reducing computational overhead while maintaining data freshness and model accuracy through systematic periodic updates
Data Source
AI summary
Systems, devices, computer-implemented methods, and/or computer program products that facilitate artificial intelligence (AI)-assisted curation of non-pharmaceutical intervention (NPI) data from heterogeneous data sources. In one example, a system can comprise a processor that executes computer executable components stored in memory. The computer executable components can comprise an extraction component and a change detection component. The extraction component can extract candidate non-pharmaceutical intervention (NPI) events from data associated with a defined disease. The change detection component can evaluate the candidate NPI events for inclusion in a dataset storing NPI events in a defined format.