AI Data Ingestion System for Multi-Format Standardization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In industries like healthcare, data ingestion from various sources in different formats is inefficient, requiring multiple processes and significant resources to analyze, as existing systems struggle to convert and process data from diverse formats, hindering the identification of relationships and trends.

Innovation Solution

A data management system utilizing machine learning models to parse, classify, and convert data into a consumable format, employing data connectors, resolvers, and feature analysis models to standardize and validate data, thereby reducing resource consumption and enabling efficient analysis.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If traditional data ingestion processes are used to handle data from multiple sources in different formats, then data can be collected, but the process requires multiple separate processes and significant resources to analyze and convert data

Engineering Contradiction:

Improveability to handle data from multiple sources in different formatsVSAvoidnumber of separate processes required

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent implements a universal data ingestion system that can handle multiple data formats and sources through a single unified process. The system uses a general-purpose parser and converter that automatically adapts to different data types (structured, semi-structured, unstructured) without requiring separate specialized processes for each format, thereby reducing overall system complexity while maintaining versatility.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent introduces an intermediary converter component that acts as a mediator between diverse data sources and the analysis system. This converter automatically transforms various data formats into a standardized internal representation, eliminating the need for multiple separate conversion processes and reducing the complexity of handling multi-format data.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If traditional data conversion processes are used, then data from diverse formats can be processed, but significant computational resources and time are consumed

Engineering Contradiction:

Improvedata format conversion capabilityVSAvoidcomputational resources consumed

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by stationary object

Solution Approach 1:

The system employs self-service automation where the data ingestion process automatically detects data formats, determines appropriate parsing methods, and performs conversion without requiring manual configuration or extensive computational resources. The automated detection and classification mechanisms enable the system to handle diverse formats efficiently using minimal processing power.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent utilizes parameter changes in the data representation to optimize processing efficiency. By transforming data into a standardized internal format with optimized data structures, the system reduces the computational complexity of subsequent analysis operations, thereby decreasing the overall resource consumption while maintaining the ability to handle multiple input formats.

Inventive Principle:

Principle #35Parameter changes

3Measurement precision

If manual data analysis processes are used, then detailed analysis can be performed, but time consumption increases and efficiency decreases

Engineering Contradiction:

Improveanalysis depth and accuracyVSAvoidtime required for data analysis

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent replaces manual mechanical data analysis processes with automated machine learning-based analysis systems. The system uses trained models to automatically detect patterns, classify data, and identify relationships, thereby maintaining high measurement precision while dramatically reducing the time required for analysis compared to manual processing methods.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system enables continuous automated data analysis that operates without interruption, processing data streams in real-time as they are ingested. This continuous processing eliminates the batch processing time delays associated with manual analysis, allowing for immediate insights while maintaining consistent analytical precision through automated algorithms.

Inventive Principle:

Principle #20Continuity of useful action

Data Source

PatentUS12124805B2Data ingestion using artificial intelligence and machine learning

Publication Date: 2024.10.22 ACCENTURE GLOBAL SOLUTIONS LTD

AI summary

A device may receive a data input that is associated with an event. The device may parse the data input to identify an input value that is associated with the event. The device may determine a probability that the input value corresponds to a feature of the event based on a configuration of the input value. The device may classify the input value as being associated with an element of the event based on the probability. The device may determine a rule profile of the input value based on the feature and the element. The device may determine a profile score associated with the data input based on the rule profile. The device may ingest, based on the profile score, the data input into a data structure. The device may determine a validation score based on a random factorization analysis of the rule profile and the input value.