AutoML Program Score Calculation via Code Injection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning programs often lack the capability to automatically calculate scores for prediction accuracy, placing a heavy burden on users to manually edit and tune AutoML systems, especially when dealing with large numbers of programs.
Innovation Solution
An information processing apparatus that rewrites machine learning programs to include code for splitting training data into training and test data, and calculates a score based on prediction data, allowing for automatic score calculation and reducing the need for manual editing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If machine learning programs are used without automatic score calculation capability, then users can use simple programs, but users bear heavy burden of manual editing and tuning
Solution Approach 1:
The machine learning program automatically calculates its own score by incorporating code that computes prediction accuracy using the trained model and test data, eliminating the need for external manual evaluation and reducing user burden
Solution Approach 2:
The program pre-calculates the score during the execution flow by inserting code before the final output stage to compute prediction accuracy using the trained model and test data, so that score calculation is completed automatically without requiring user intervention later
2Measurement precision
If manual editing of machine learning programs is required for score calculation, then score accuracy can be ensured, but time and effort are significantly consumed
Solution Approach 1:
The manual mechanical process of editing programs to add score calculation is replaced by an automatic code insertion mechanism that detects training function calls and injects the necessary prediction and scoring code, maintaining accuracy while eliminating time-consuming manual work
Solution Approach 2:
An intermediary code insertion mechanism acts as a bridge between the original machine learning program and the score calculation requirement, automatically detecting training functions and inserting the appropriate prediction and evaluation code without requiring users to manually edit the program
3Productivity
If machine learning programs do not include automatic score calculation, then program complexity is reduced, but productivity in tuning AutoML systems decreases
Solution Approach 1:
The program automatically divides the data processing flow into distinct segments: training data preparation, model training, test data prediction, and score calculation, making the AutoML tuning process more efficient through structured automated evaluation
Solution Approach 2:
The automatically calculated score provides immediate feedback on model performance, enabling efficient AutoML tuning by allowing systematic comparison of different programs and automated selection of high-performing models without manual intervention
Data Source
AI summary
A computer detects, based on a module name of a module included in a machine learning library, a first code for calling a training function for training a machine learning model from a machine learning program. The computer inserts, before the first code, second code for splitting first training data passed to the training function into second training data and test data. The computer rewrites the first code to pass the second training data to the training function. The computer inserts, after the first code, third code for calculating a score based on prediction data, which is generated from the trained machine learning model and input data included in the test data, and correct answer data included in the test data.