Meta-Database Profiling for Faster AI Training Across Data Silos
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI systems face challenges in identifying relevant data across disparate and heterogeneous sources, leading to increased processing time, power consumption, and reduced efficiency in model training due to the difficulty in locating and utilizing important variables and data stored in separate databases.
Innovation Solution
A meta-database system that utilizes AI algorithms to scan, categorize, and profile data from multiple sources, generating associations and probability distributions to identify critical variables, reducing the need for extensive data processing by creating a summarized dataset.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If additional input data is provided to an AI system, then the accuracy of the generated output is improved, but the processing time and computing power consumption increase
Solution Approach 1:
The patent applies preliminary action by creating a meta-database that pre-processes and organizes data from multiple heterogeneous sources before the actual AI model training. The system performs preliminary data scanning, categorization, and relationship identification across disparate databases, storing this processed information in a meta-database structure that enables rapid retrieval. This preliminary organization of data relationships and structures allows the AI system to access only relevant pre-processed data during training, rather than searching through all raw data, thus maintaining accuracy while reducing processing time.
2Measurement precision
If additional input data is provided to an AI system, then the accuracy of the generated output is improved, but the computing power consumption increases
Solution Approach 1:
The patent applies the extraction principle by selectively identifying and extracting only the most relevant data and relationships from extensive heterogeneous databases. The meta-database system scans multiple data sources, identifies key variables and their relationships, and extracts this essential information into a condensed meta-database structure. During AI model training, only this extracted relevant data is utilized, rather than processing all available data from the original heterogeneous sources. This selective extraction maintains model accuracy while significantly reducing computing power consumption.
3Ease of manufacture
If data is stored in distinct and separate digital locations, then data organization is improved, but the difficulty of locating relevant data increases
Solution Approach 1:
The patent applies the intermediary principle by introducing a meta-database as a mediating layer between the original heterogeneous data sources and the AI system. The meta-database receives data from multiple separate digital locations, processes and standardizes the information, and stores it in a unified structure with identified relationships between variables. This intermediary meta-database maintains the organizational benefits of separate data sources while providing a centralized access point that eliminates the difficulty of locating relevant data across disparate systems. The AI system queries the meta-database rather than directly accessing multiple separate sources, thus simplifying data location and access.
Data Source
AI summary
A system for training a model from a subset of data representing decentrally stored source databases. A key variable repository module operably couples the databases and includes an AI program with a scanner algorithm and a profiler algorithm. The scanner algorithm receives the training data from a source interface, compresses the training data, and synchronizes the training data with the meta-data using a meta-database interface. The profiler algorithm receives the meta-data from the meta-database interface, generates granular data types for the meta-data, determines training variables indicative of the meta-data, generates variable probability distributions, produces training variable associations, and modifies the meta-database to include the probability distributions and associations using the meta-data interface. The key interface allows for searching the meta-database for training variables, variable probability distributions, and/or variable associations. A model of the system may be trained in less time with a subset of data associated with the training variable.