ML Platform Structuring Unstructured Data via Ontology Standardization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Enterprise data processing systems face challenges in utilizing machine learning techniques on unstructured data, as existing methods struggle to effectively process semi-structured and unstructured data formats such as images, health records, and metadata, which are not directly compatible with machine learning models.
Innovation Solution
A machine learning platform that employs source-agnostic models to preprocess unstructured data, optimizing inputs for machine learning models by segmenting large documents, applying image quality enhancement, optical character recognition, and using reference ontologies to standardize data attributes and improve accuracy, particularly in identifying medication entities with spelling irregularities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If unstructured data is directly input into machine learning models, then the processing speed is maintained, but the model performance and accuracy deteriorate because unstructured data formats are not compatible with machine learning models
Solution Approach 1:
The system performs preliminary preprocessing of unstructured data before it reaches the machine learning model. This includes segmenting large documents into smaller units, enhancing image quality, performing optical character recognition, and extracting relevant information in advance, so that the model receives pre-processed structured data ready for analysis
Solution Approach 2:
The system introduces an intermediary preprocessing layer between the unstructured data source and the machine learning model. This intermediary component transforms unstructured data into structured formats through various processing steps including segmentation, image enhancement, OCR, and information extraction, making the data compatible with model input requirements
2Measurement precision
If unstructured data is preprocessed to improve model accuracy, then classification accuracy improves, but processing time increases
Solution Approach 1:
The system segments large unstructured documents into smaller, manageable units before processing. This segmentation allows the preprocessing operations (image enhancement, OCR, information extraction) to be performed on smaller chunks in parallel, reducing overall processing time while maintaining the quality improvements necessary for accurate classification
3Manufacturing precision
If reference ontologies are used to standardize data attributes, then data accuracy improves, but system complexity increases
Solution Approach 1:
The system changes the parameters of data attributes by standardizing them according to reference ontologies. This transformation converts varied and inconsistent data formats into standardized parameters that are consistent with domain knowledge, improving data accuracy and reliability for machine learning processing
Data Source
AI summary
An application instance that includes one or more machine learning models receives, from a subscriber computing system, a document comprising unstructured data. Based on the unstructured data, the application instance generates an optimized model input that includes a plurality of parsed document sections. For each parsed document section, the application instance generates an output set by performing, by a machine learning model, at least one key information extraction operation. The machine learning model transmits the output in structured form to a target application operated or hosted at least in part by a subscriber entity associated with the subscriber computing system.