AI Model Deployment Using a Unified Metadata Graph
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems require significant computational resources and manual effort to access siloed data across disparate locations, leading to inefficiencies, data integrity issues, and wasted memory due to duplicate data storage, while existing solutions to create unified data silos result in system downtimes and further resource waste.
Innovation Solution
A unified metadata graph system that uses natural language processing and Large Language Models (LLMs) to generate a domain-specific metadata graph, reducing the need for new data silos and reconfiguration, by determining data locations and lineages across silos, and optimizing metadata generation to preserve data integrity and reduce retrieval times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data silos are created for each computing system to process data efficiently, then data processing efficiency is improved, but compute capacity requirements increase due to duplicate data storage and maintenance
Solution Approach 1:
The patent introduces a data catalog as an intermediary layer between computing systems and data silos. This catalog maintains a unified view of data locations and characteristics across all silos, enabling efficient data discovery and access without requiring compute resources to process duplicate data copies. The catalog acts as a mediator that resolves data redundancy while preserving the efficiency benefits of distributed data processing.
2Ease of operation
If new data silos are created to consolidate data, then data accessibility is improved, but system downtime increases and computational resources are wasted
Solution Approach 1:
The patent implements preliminary action by pre-building and maintaining a comprehensive data catalog that continuously inventories all data silos before consolidation is needed. This catalog is updated in real-time as data moves between silos, so when data accessibility is required, the system can immediately query the existing catalog without initiating time-consuming consolidation operations. The preliminary catalog structure eliminates the need for disruptive reorganization.
3Reliability
If data is stored in multiple data silos with unique variable names and formatting, then data integrity for each system is maintained, but data retrieval complexity increases
Solution Approach 1:
The patent applies segmentation by separating data integrity concerns from data access operations. Each data silo maintains its own variable names and formatting conventions for integrity purposes, while the data catalog segments the access layer into standardized queries. Users interact with the catalog using unified data definitions, while the catalog handles the complexity of mapping these to the actual silo-specific formats, preserving both integrity and simplicity.
Data Source
AI summary
A system facilitates a process for automatically deploying artificial intelligence (AI) models. The system receives, for a first artificial intelligence (AI) model used by an entity, a first request to deploy the first AI model to make the first AI model available for use in a production environment to process input data and generate corresponding outputs. A first model deployment location for the first model is selected based on a model deployment engine. The system generates scripts to deploy the first AI model to the selected location, then monitors operations parameters associated with the deployment of the first AI model. Based on the values of the operations parameters, the system updates the model deployment engine. In response to a second request to deploy a second AI model, the system uses the updated model deployment engine to select a second model deployment location for the second model.