Constraint-Based Hyperparameter Tuning Across Multiple Metrics

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Standard hyperparameter tuning algorithms for chatbot systems focus on a single goal, ignoring other important objectives and considering all datasets equally, leading to suboptimal performance across diverse metrics and datasets.

Innovation Solution

A hyperparameter tuning system that optimizes multiple metrics simultaneously, assigns weights to each metric and dataset importance, and incorporates constraints to train a machine-learning model effectively across different datasets and objectives.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If standard hyperparameter tuning algorithms optimize for a single goal, then model accuracy for that specific goal is improved, but performance across other metrics and datasets deteriorates

Engineering Contradiction:
Improvemodel accuracyVSAvoidperformance across multiple metrics and datasets
Core Design Contradiction:
Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent creates a unified hyperparameter tuning system that simultaneously optimizes for multiple goals and datasets rather than training separate models for each goal. The system uses a composite objective function that aggregates performance across multiple metrics (accuracy, precision, recall, F1-score) and multiple datasets, allowing a single model to achieve balanced performance across diverse evaluation criteria. This resolves the contradiction by making the tuning algorithm itself multi-functional, capable of optimizing for both specific accuracy targets and broad adaptability simultaneously.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent introduces weighted parameters for different metrics and datasets in the objective function. By adjusting these weights, the system can dynamically balance the optimization focus between different goals and datasets. This parameter-based control allows the same tuning algorithm to adapt to different performance requirements without sacrificing either single-goal accuracy or multi-goal versatility, as the weight parameters enable flexible prioritization while maintaining comprehensive optimization.

Inventive Principle:
Principle #35Parameter changes

2Measurement precision

If multiple models are trained for different goals, then each model achieves optimal performance for its specific goal, but system complexity increases

Engineering Contradiction:
Improvegoal-specific performanceVSAvoidnumber of models to train and maintain
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent merges multiple objective functions into a single composite objective function that incorporates multiple goals and datasets. Instead of training separate models for different goals, the system combines accuracy, precision, recall, and F1-score metrics into one unified optimization target. This merging approach maintains the performance benefits of goal-specific optimization while reducing system complexity by requiring only a single model training process, thereby resolving the contradiction between specialized performance and system simplicity.

Inventive Principle:
Principle #5Merging (Combining)

Solution Approach 2:

The unified hyperparameter tuning system serves multiple goals simultaneously through a single model, making the tuning process universal rather than goal-specific. The system evaluates performance across multiple metrics and datasets using one model, eliminating the need to maintain multiple specialized models. This multi-functional approach achieves both optimal goal-specific performance and reduced complexity by consolidating what would otherwise require separate modeling efforts into one comprehensive tuning framework.

Inventive Principle:
Principle #6Universality (Multi-functionality)

3Ease of manufacture

If all datasets are treated equally in hyperparameter tuning, then the tuning process is simple, but performance on specific important datasets deteriorates

Engineering Contradiction:
Improvetuning process simplicityVSAvoidperformance on important datasets
Core Design Contradiction:
Ease of manufactureVSMeasurement precision

Solution Approach 1:

The patent applies different weights to different datasets in the objective function, allowing important datasets to have greater influence on the tuning process. Instead of treating all datasets uniformly, the system assigns local quality parameters (weights) that reflect the relative importance of each dataset. This enables the tuning process to focus computational effort on critical datasets while maintaining a relatively simple weighted aggregation approach, resolving the contradiction between process simplicity and targeted performance optimization.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The system uses weight parameters to control the relative importance of different datasets and metrics in the objective function. By adjusting these parameters, the tuning process can prioritize specific important datasets without fundamentally changing the simplicity of the weighted optimization approach. This parameter-based differentiation allows the system to maintain ease of implementation while achieving superior performance on critical datasets through controlled parameter adjustment rather than complex processing.

Inventive Principle:
Principle #35Parameter changes

Data Source

PatentUS12405975B2Method and system for constraint based hyperparameter tuning
Publication Date: 2025.09.02 ORACLE INT CORP
  • US12405975B2 patent drawing
  • US12405975B2 patent drawing
  • US12405975B2 patent drawing

AI summary

Techniques are disclosed for tuning hyperparameters of a model. Datasets are obtained for training the model and metrics are selected for evaluating performance of the model. Each metric is assigned a weight specifying an importance to the performance of the model. A function is created that measures performance based on the weighted metrics. Hyperparameters are tuned to optimize the model performance. Tuning the hyperparameters includes: (i) training the model that is configured based on a current values for the hyperparameters; (ii) evaluating a performance of the model using the function; (iii) determining whether the model is optimized for the metrics; (iv) in response to the model not being optimized, searching for a new values for the hyperparameters, reconfiguring the model with the new values, and repeating steps (i)-(iii) using the reconfigured model; and (v) in response to the model being optimized for the metrics, providing a trained model.