Functional Block Labeling for Non-Expert Machine Learning Pipeline Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AutoML solutions are insufficient for non-experts to fully implement new machine learning projects due to a scarcity of ML experts, and open-source software databases like Kaggle and GitHub are difficult for non-experts to navigate and modify for new project requirements.
Innovation Solution
A computer system automatically searches and adapts existing ML projects by curating high-quality and diverse projects from OSS databases, identifying and labeling functional blocks, and synthesizing a new ML pipeline using a hierarchical approach, enabling non-experts to generate a new ML pipeline without manual modification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If non-experts use current AutoML solutions, then they can partially automate ML项目实施, but the solutions are insufficient to enable full implementation of new ML projects
Solution Approach 1:
The patent segments ML pipelines into discrete functional blocks (e.g., data loading, preprocessing, model training, evaluation) that can be independently selected and combined. This segmentation enables non-experts to build complete ML projects by assembling pre-labeled functional blocks from a corpus, rather than implementing entire pipelines manually or relying on insufficient AutoML solutions.
Solution Approach 2:
The patent performs preliminary actions by pre-labeling functional blocks in a corpus of existing ML projects with semantic tags indicating their purpose and functionality. This pre-processing enables non-experts to quickly search and assemble appropriate blocks for new projects without having to understand or manually analyze the underlying code, thereby enabling full project implementation.
2Quantity of substance
If non-experts navigate open-source software databases like Kaggle and GitHub, then they can access existing ML projects, but it is difficult for them to modify projects for new requirements
Solution Approach 1:
The patent introduces an intermediary system that acts as a mediator between non-experts and open-source ML projects. This system automatically curates, parses, and labels functional blocks from raw ML projects in databases like Kaggle and GitHub, then presents them in an accessible format with semantic annotations. Non-experts can search and assemble these labeled blocks without directly navigating or modifying the original complex codebases.
Solution Approach 2:
The patent creates simplified copies of functional blocks from existing ML projects, extracting and labeling only the essential components needed for new projects. Instead of requiring non-experts to work with complete, complex original projects, the system provides copied, modular functional blocks that can be easily selected and combined for new requirements.
3Quantity of substance
If ML experts are scarce, then the availability of skilled data scientists is limited, but the demand for implementing new ML projects is increasing
Solution Approach 1:
The patent enables self-service by allowing non-experts to independently implement complete ML projects using the automated system. The system automatically curates relevant functional blocks from a corpus, labels them with semantic information, and enables non-experts to assemble pipelines without requiring ML expert intervention, thereby increasing project implementation rate despite limited expert availability.
Solution Approach 2:
The patent changes the parameter of skill requirement from high (requiring ML expertise) to low (accessible to non-experts) by automating the complex tasks of project curation, functional block extraction, and semantic labeling. This parameter change enables a much larger population to implement ML projects, increasing overall productivity.
4Adaptability or versatility
If the corpus of ML projects is large, then the diversity and quality of available projects increase, but the complexity of searching and adapting projects increases
Solution Approach 1:
The patent replaces the mechanical process of manual searching and analyzing ML projects with an automated computational system. The system automatically curates projects from the corpus, extracts functional blocks, labels them with semantic information, and enables efficient searching based on project requirements, thereby managing the complexity of large diverse corpora without requiring manual effort.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
According to one or more embodiments, operations may include normalizing machine learning (ML) pipelines of existing ML projects stored in a corpus of existing ML projects. The operations may also include extracting functional blocks from the normalized ML pipelines. In addition, the operations may include assigning a label to each of the functional blocks in the normalized ML pipelines. Moreover, the operations may include indexing each of the ML pipelines in the corpus based on the labels assigned to the functional blocks. In addition, the operations may include utilizing the labels assigned to the functional blocks in the corpus to generate a new pipeline to perform a new ML task on a new dataset of a new ML project.