ML Pipeline Data Versioning via Model Evaluation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning pipeline technologies face inefficiencies in training data version management, leading to storage capacity issues due to redundant data, difficulty in comparing model creation conditions, and the need for separate source code implementation, which results in coding errors and wasted time.
Innovation Solution
A management device with a data processor, manager, and evaluator that performs preprocessing, saves training datasets, and determines permanent storage based on model evaluation results, integrating version management within the machine learning pipeline to eliminate redundancy and streamline the process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If training datasets are saved every time a model is created, then model version management is improved, but storage capacity is consumed due to redundant data
Solution Approach 1:
The patent merges version management functionality directly into the machine learning pipeline components. The management device integrates with the pipeline to automatically manage training data versions without requiring separate external version control systems, thereby reducing redundant storage while maintaining reliable version tracking.
Solution Approach 2:
The management device performs multiple functions including training data management, model evaluation, and version control within a single integrated system. This multi-functional approach eliminates the need for separate dedicated systems for each function, reducing overall storage requirements while improving version management reliability.
2Adaptability or versatility
If separate source code is used for version management, then version control flexibility is improved, but coding errors and development time increase
Solution Approach 1:
The patent combines version management capabilities with the machine learning pipeline execution flow. The management device automatically handles version control as part of the pipeline execution, eliminating the need for separate source code implementations and reducing development time while maintaining flexibility.
Solution Approach 2:
The machine learning pipeline performs version management automatically through the integrated management device without requiring manual coding or external intervention. The system self-manages training data versions and model evaluations, reducing development time and potential coding errors while maintaining adaptability.
3Loss of information
If all training data is permanently saved, then data availability is improved, but storage pressure increases
Solution Approach 1:
The patent applies different quality levels to different training data based on their importance and usage frequency. The management device selectively retains training data locally based on evaluation results and model performance, ensuring critical data remains readily available while reducing storage pressure for less important data.
Solution Approach 2:
The system selectively discards training data that is no longer needed based on model evaluation outcomes, and recovers or retains data that proves valuable for future model iterations. This selective retention strategy maintains data availability for important cases while reducing overall storage pressure.
Data Source
AI summary
According to an embodiment, a management device includes a data processor, a data manager, and an evaluator. The data processor is configured to perform at least one preprocessing operation of creating a training dataset. The data manager is configured to perform a process of saving the created training dataset. The evaluator is configured to evaluate a model created using the created training dataset. The data manager is configured to temporarily save the created training dataset, and determine whether or not to permanently save the created training dataset on the basis of an evaluation result of the model by the evaluator.