Automated Hyperparameter Tuning via Application History

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing hyperparameter tuning methods for machine learning models lack efficiency in setting appropriate hyperparameters, leading to suboptimal predictive performance and learning efficiency, as they often require manual setting and lack automated processes for determining optimal parameter combinations.

Innovation Solution

A hyperparameter tuning method that uses a hyperparameter obtaining code within the user program to request and obtain hyperparameters based on an application history, employing algorithms like Bayesian optimization and random search to iteratively determine and optimize hyperparameter settings, facilitating automated hyperparameter tuning.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If manual hyperparameter setting is used, then ease of operation is maintained, but productivity and predictive performance deteriorate due to time-consuming trial and error

Engineering Contradiction:

Improvehyperparameter tuning efficiencyVSAvoidtime for hyperparameter setting

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The system performs automated hyperparameter tuning by having the program itself request and obtain appropriate hyperparameters based on its application history, eliminating the need for manual intervention and significantly reducing time loss while improving tuning efficiency

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system pre-obtains and stores hyperparameters in an application history database before actual program execution, allowing rapid retrieval and application of optimized parameters without time-consuming manual adjustment during runtime

Inventive Principle:

Principle #10Preliminary action

2Productivity

If automated hyperparameter tuning is implemented, then productivity improves, but device complexity increases due to additional systems and algorithms

Engineering Contradiction:

Improvehyperparameter tuning efficiencyVSAvoidsystem complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The hyperparameter obtaining system is designed to be universally applicable across different programs and domains by using a standardized interface where programs simply request hyperparameters based on their application history, allowing the same mechanism to serve multiple functions without increasing complexity

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

An intermediary hyperparameter obtaining system is introduced that mediates between the program and the hyperparameter optimization process, managing the complexity of optimization algorithms internally while presenting a simple interface to user programs

Inventive Principle:

Principle #24Intermediary (Mediator)

3Reliability

If appropriate hyperparameters are set, then predictive performance improves, but ease of operation deteriorates due to the complexity of parameter selection

Engineering Contradiction:

Improvepredictive performanceVSAvoidease of hyperparameter setting

Core Design Contradiction:

ReliabilityVSEase of operation

Solution Approach 1:

The program automatically requests and obtains appropriate hyperparameters based on its own application history and performance requirements, eliminating the need for user intervention in complex parameter selection while ensuring high predictive performance through optimized parameter choices

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS20210224692A1Hyperparameter tuning method, device, and program

Publication Date: 2021.07.22 PREFERRED NETWORKS INC

AI summary

A hyperparameter tuning method for execution by one or more processors includes receiving a request to obtain a hyperparameter, the request being generated according to a hyperparameter obtaining code, and the hyperparameter obtaining code being written in a user program, and providing the hyperparameter to the user program based on an application history of hyperparameters applied to the user program.