OCI Model Repository for Layered AI Deployment Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional machine learning models, particularly Large Language Models (LLMs), are complex and massive, making them difficult to download and integrate into customer computing environments due to network limitations, which restricts their use in private, secured networks and introduces security concerns with remote system access.
Innovation Solution
The implementation of a machine learning model staging repository using open container images (OCIs) that store models as separate layers, allowing for automated downloads, integration, and updates, optimizing storage and simplifying the process by maintaining a directory structure for efficient bandwidth use.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If machine learning models are stored and transferred as single large files, then the model can be complete and functional, but the download time becomes excessively long and network bandwidth is inefficiently utilized
Solution Approach 1:
The patent segments the machine learning model into multiple layers stored as separate files in a directory structure. Each layer can be downloaded independently, allowing parallel transfer and reducing overall download time. The model is divided into manageable chunks that can be processed and integrated incrementally at the customer site.
2Productivity
If machine learning models are downloaded completely before use, then the model is available for deployment, but network bandwidth is consumed inefficiently and download time increases
Solution Approach 1:
The patent enables preliminary downloading of model layers that are most likely to be needed first, allowing the system to prepare model components in advance without requiring complete model download. Critical layers can be staged locally while less urgent layers remain available for on-demand retrieval.
3Reliability
If machine learning models are updated by re-downloading the entire model, then the model is kept current, but excessive bandwidth is consumed and update time increases
Solution Approach 1:
The patent enables selective updating of individual model layers by downloading only the updated layer files rather than re-downloading the entire model. The directory structure allows identification of which layers have changed, and only those specific layers are transferred, significantly reducing update bandwidth consumption.
4Adaptability or versatility
If machine learning models are accessed remotely from a model provider, then the customer does not need to store large model files locally, but security concerns arise from transferring data over open networks and sharing remote systems
Solution Approach 1:
The patent extracts the model layers from the remote model provider system and stores them locally in a directory structure at the customer site. This extraction eliminates the need for continuous network access to the remote system, reducing security risks associated with open network transfers and shared remote access while maintaining model functionality.
Data Source
AI summary
Systems and methods are provided for creating a machine learning (ML) model staging repository, where ML models are stored as an open container image (OCI). The OCI comprises layers or portions of the complete ML model, so that the model can be stored separately and as a smaller files. The ML models may be pre-packaged for automated downloads and integration at the customer site. In some examples, the OCI can identify/store the model in a directory structure that defines the model and its profile. The OCI can comprise a combination of layers and profiles that allows the AI platform to optimize the storage and simplify the process of downloading or updating the given user namespace instead of downloading a single large file for the model.