Machine Learning Model Data Tiering With Version-Difference Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face inefficiencies in managing flash storage devices, particularly in the management of erase blocks and data operations, leading to unnecessary write operations and reduced reliability.
Innovation Solution
A direct-mapped flash storage system where the operating system directly addresses data blocks without translation by storage controllers, initiating and controlling processes such as data rewriting and erasure, thereby offloading device management responsibilities from individual flash drives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If storage controllers translate and manage data blocks in flash storage devices, then data access can be coordinated, but redundant write operations increase and reliability decreases
Solution Approach 1:
The patent extracts the data block translation and management functions from the storage controllers and relocates them to the operating system. This separation eliminates the redundant translation operations that were causing reliability issues and unnecessary write operations, as the OS directly manages data blocks without controller intervention.
Solution Approach 2:
The operating system takes on the responsibility of directly addressing and managing data blocks in flash storage devices without requiring storage controller translation. This self-service approach allows the OS to directly initiate and control data rewriting and erasure processes, eliminating the inefficiencies of controller-mediated management.
2Ease of operation
If storage controllers manage data block translation, then data access coordination is enabled, but device management complexity increases
Solution Approach 1:
The patent extracts the data block translation and management functions from the storage controllers and relocates them to the operating system. This separation eliminates the redundant translation operations that were causing reliability issues and unnecessary write operations, as the OS directly manages data blocks without controller intervention.
3Adaptability or versatility
If flash storage devices individually manage their own erase blocks, then device autonomy is maintained, but overall system efficiency decreases
Solution Approach 1:
The patent merges the erase block management responsibilities from individual flash storage devices into a unified operating system-level management system. This consolidation allows for coordinated control of data rewriting and erasure across multiple devices, eliminating redundant operations and improving overall system efficiency while maintaining the autonomy of individual devices through OS-managed allocation.
Data Source
AI summary
Improving machine learning models in an artificial intelligence infrastructure includes: storing, within one or more storage systems of an artificial intelligence infrastructure, information describing a dataset and one or more transformations applied to the dataset resulting in a transformed dataset; and storing, within the one or more storage systems, information describing only portions of previous versions of a machine learning model that differ from a current version of the machine learning model, wherein the previous versions used the transformed dataset as input during one or more prior executions by the artificial intelligence infrastructure.