Metadata-Driven AI Platform for Model Portability and Auditability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning and artificial intelligence technologies face challenges in making AI models portable, reusable, and auditable due to the lack of containerization of training data and provenance information, which complicates the updating and tracking of AI models.
Innovation Solution
A metadata-driven AI platform is introduced, utilizing a set of metafiles that store metadata and provenance information, along with an API to manage AI processes and datasets, allowing for the recreation of execution environments and tracking of AI model lineage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If AI models are trained and deployed without containerization of training data and provenance information, then the training and deployment process is simpler, but the models become difficult to update, track, and audit
Solution Approach 1:
The patent segments the AI process into distinct containerized components: training data, provenance information, metadata, and AI models are each encapsulated in separate containers. This segmentation enables independent management, tracking, and auditing of each component while maintaining overall system coherence through the metadata-driven orchestration layer.
Solution Approach 2:
The patent introduces a metadata-driven platform as an intermediary layer between the AI processes and the underlying infrastructure. This intermediary manages containerization, tracks provenance information, and orchestrates the lifecycle of AI models, thereby improving auditability without directly increasing the complexity of individual AI components.
2Adaptability or versatility
If training data and provenance information are not containerized, then the storage and management process is simpler, but the portability and reusability of AI models are reduced
Solution Approach 1:
The patent creates universal container structures that can hold training data, provenance information, and metadata in a standardized format. These containers serve multiple functions: they enable portability across different systems, support reusability of training datasets, and provide a unified interface for managing diverse AI workloads, thereby improving adaptability without proportionally increasing complexity.
Solution Approach 2:
The patent applies preliminary containerization to training data and provenance information before they are used in AI training processes. By pre-organizing these elements into standardized containers with embedded metadata, the system enables easier portability and reusability during deployment while the complexity of containerization is resolved in advance rather than during runtime.
3Loss of information
If AI processes are managed without a metadata-driven approach, then the system operation is simpler, but the ability to track and audit AI model lineage is compromised
Solution Approach 1:
The patent implements a nested structure where provenance information and metadata are embedded within container objects that encapsulate training data and AI models. This nesting ensures that provenance information travels with the data it describes, preventing information loss during transfer and deployment while the hierarchical organization manages complexity through structured encapsulation.
Solution Approach 2:
The patent creates metadata copies that are embedded within container structures alongside the original training data and provenance information. These metadata copies enable tracking and auditing without requiring access to the original data sources, thereby preventing information loss while managing complexity through self-contained container units that can be independently processed.
Data Source
AI summary
A set of metafiles that stores at least metadata information and provenance information of an artificial intelligence (AI) process is generated, where the AI process is trained with a source data. The set of metafiles is accessed via an application programming interface (API) to the set of metafiles. In response to accessing the set of metafiles, the source data in the set of metafiles is transferred to a cache for processing by the AI process.