ML Pipeline Skeleton Instantiation via Code Snippet Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AutoML solutions and open-source software databases are insufficient for non-expert data scientists to implement new machine learning projects due to scarcity of ML experts, and existing ML projects are difficult to adapt for new requirements.
Innovation Solution
A computer system that automatically adapts existing ML projects by curating and indexing functional blocks from OSS databases, allowing non-experts to search and adapt these blocks into new ML pipelines using a 'search-and-adapt' workflow, with techniques like dynamic program slicing and graph-based sequence mining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing ML projects are used directly, then implementation speed is improved, but adaptability to new requirements deteriorates
Solution Approach 1:
The patent segments existing ML projects into reusable functional blocks (e.g., data loading, preprocessing, model training) that can be independently selected and adapted. This allows users to compose new pipelines from proven components, maintaining productivity while enabling adaptability through modular recombination.
Solution Approach 2:
The system enables parameter changes by allowing users to modify hyperparameters, data sources, and configuration settings of functional blocks without changing the underlying code structure. This resolves the contradiction by maintaining the proven logic of existing projects while adapting to new requirements through parameter adjustment.
2Ease of operation
If AutoML solutions are used, then ease of operation is improved, but manufacturing precision deteriorates
Solution Approach 1:
The patent copies proven functional blocks from high-quality existing ML pipelines into new projects. By reusing validated code segments from expert implementations, the system maintains manufacturing precision while improving ease of operation, as users can leverage pre-tested components without needing deep ML expertise.
Solution Approach 2:
The system creates universal functional blocks that can serve multiple purposes across different ML projects. These multi-functional components maintain high quality through rigorous testing while being easily operable by non-experts through standardized interfaces and configuration options.
3Adaptability or versatility
If manual adaptation of existing projects is performed, then adaptability is improved, but loss of time increases
Solution Approach 1:
The patent applies preliminary action by pre-processing existing ML projects into standardized functional blocks with documented interfaces and dependencies. This upfront preparation enables rapid adaptation to new requirements without time-consuming manual modification, as the decomposition work is done once and reused across multiple projects.
4Ease of operation
If OSS databases are curated and indexed, then ease of operation is improved, but device complexity increases
Solution Approach 1:
The system applies self-service by implementing automated curating and indexing mechanisms that organize functional blocks from OSS databases without requiring manual intervention. The system automatically extracts, categorizes, and indexes code segments, improving ease of operation while managing complexity through automation rather than manual processes.
Data Source
AI summary
Operations include obtaining a machine learning (ML) pipeline skeleton that indicates a set of first functional blocks to use to process a new dataset of a new ML project. Additionally, for each respective first functional block of the set of first functional blocks, the operations include obtaining existing code snippets from existing ML pipelines, each of the existing code snippets instantiating a second functional block of the existing ML pipelines and being a potential instantiation of the respective first functional block. The operations also include determining a respective adaptability, with respect to the new dataset, of each of the existing code snippets and selecting a particular existing code snippet for implementation of the respective first functional block based on the determined adaptabilities. Further, the operations include instantiating the pipeline skeleton based on the particular existing code snippets.