Machine Learning Support System Prioritizing Candidate Programs by User Proficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users face difficulties in editing machine learning pipelines generated by AutoML systems, as they often include unfamiliar packages, leading to increased time and effort in improving model accuracy.
Innovation Solution
A machine learning support system that prioritizes and presents candidate programs based on user proficiency levels, using proficiency level information to determine the ease of editing and accuracy of program components, thereby facilitating efficient editing and improving model performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If AutoML systems generate candidate pipelines using diverse packages to improve model accuracy, then model performance is improved, but user editing difficulty increases due to unfamiliar packages
Solution Approach 1:
The system performs preliminary action by determining user proficiency levels for different packages before presenting candidate pipelines. This allows the system to pre-filter and prioritize pipelines using packages the user is already familiar with, thereby reducing editing difficulty while maintaining model accuracy performance.
Solution Approach 2:
The system applies local quality by differentiating the presentation strategy for different candidate pipelines based on local characteristics - specifically, the proficiency level of the user for each package used in the pipeline. Pipelines are individually assessed and prioritized according to their compatibility with user expertise in specific packages.
2Adaptability or versatility
If AutoML systems present all candidate pipelines to users for selection, then user choice and potential for improvement is maximized, but time required for user evaluation increases
Solution Approach 1:
The system performs preliminary filtering and prioritization of candidate pipelines based on user proficiency levels before presenting them to the user. This reduces the number of pipelines the user needs to evaluate manually, significantly cutting evaluation time while preserving access to high-quality options through the prioritized list.
Solution Approach 2:
The system incorporates feedback mechanisms by tracking user editing behavior and package usage patterns to dynamically update proficiency level assessments. This feedback loop enables the system to continuously improve its prioritization accuracy, presenting increasingly relevant pipelines that match user expertise, thereby reducing evaluation time over repeated interactions.
3Reliability
If AutoML systems use unfamiliar packages in generated pipelines to achieve higher performance, then model capability is improved, but learning curve and editing effort for users increases
Solution Approach 1:
The system performs preliminary assessment of user proficiency for various packages and uses this information to prioritize candidate pipelines that align with user expertise. This allows the system to balance model capability with editing effort by presenting pipelines that achieve high performance using packages the user already knows, thereby reducing the learning curve and editing effort required.
Data Source
AI summary
A process includes receiving, by a machine learning support system, an instruction to generate a machine learning model from a plurality of candidate-programs, specifying, for each of the plurality of candidate-programs generated using a program-component included in any of a plurality of program-component sets, a first proficiency-level of a user for a first program-component set which includes a first program-component used in the candidate-program, the first proficiency-level is based on proficiency-level information which indicates a proficiency-level of the user related to use of each of the plurality of program-component sets and is determined based on a use record of the plurality of program-component sets in an editing process of the candidate-program by the user and a change in performance of the candidate-program by the editing process, and determining, for each of the plurality of candidate-programs, a priority to present the candidate-program to the user, based on the specified first proficiency-level.