Neural Network Hyperparameter Optimization via Iterative Re-evaluation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional methods for determining hyperparameter combinations in neural networks are inefficient and inaccurate, leading to suboptimal performance due to limited evaluation and potential errors in hyperparameter selection.

Innovation Solution

A neural network optimization method that involves sampling the hyperparameter search space, performing iterative evaluations, re-evaluating hyperparameter combinations that meet specific conditions, and updating a preset model based on performance results to improve the accuracy of hyperparameter selection.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If conventional hyperparameter evaluation methods are used with limited iterations, then evaluation time is reduced, but measurement precision of hyperparameter performance is insufficient

Engineering Contradiction:

Improvehyperparameter performance evaluation accuracyVSAvoidhyperparameter evaluation time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent performs preliminary actions by conducting multiple iterative evaluations and re-evaluations of hyperparameter combinations before final selection. The system evaluates each hyperparameter combination multiple times, re-evaluates promising combinations after model updates, and only then determines the optimal hyperparameter combination, ensuring high measurement precision without excessive time loss.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent implements feedback mechanisms by using performance results from evaluations to update the preset model, then using the updated model to guide subsequent evaluations. The system re-evaluates hyperparameter combinations based on feedback from previous evaluation rounds, progressively improving measurement precision while managing evaluation time through intelligent sampling.

Inventive Principle:

Principle #23Feedback

2Reliability

If hyperparameter combinations are evaluated only once, then evaluation process is simplified, but reliability of evaluation results is insufficient

Engineering Contradiction:

Improvehyperparameter evaluation result reliabilityVSAvoidevaluation process complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent performs preliminary evaluations of hyperparameter combinations before final selection. Each combination is evaluated multiple times in advance, and promising combinations are re-evaluuated after model updates, ensuring reliable results before the final hyperparameter determination is made.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent applies partial evaluation by focusing computational resources on re-evaluating only those hyperparameter combinations that meet specific conditions (promising combinations) rather than evaluating all combinations equally. This excessive action on selected combinations ensures high reliability without requiring complete re-evaluation of all possibilities.

Inventive Principle:

Principle #16Partial or excessive action

3Measurement precision

If the preset model is updated frequently with new performance data, then probability distribution accuracy is improved, but computational overhead increases

Engineering Contradiction:

Improveprobability distribution fitting accuracyVSAvoidcomputational energy consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent performs preliminary model updates by incorporating performance results into the preset model after each evaluation round. The model is updated before subsequent sampling and evaluation iterations, ensuring accurate probability distributions guide the search process without requiring constant updates during every operation.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent implements periodic model updates rather than continuous updates. The preset model is updated at specific intervals based on accumulated performance results from evaluation iterations, balancing probability distribution accuracy with computational efficiency by updating only when sufficient new data is available.

Inventive Principle:

Principle #19Periodic action

Data Source

PatentUS12327400B2Neural network optimization method and apparatus

Publication Date: 2025.06.10 HUAWEI TECH CO LTD

AI summary

The present disclosure relates to neural network optimization methods and apparatuses in the field of artificial intelligence. One example method includes sampling preset hyperparameter search space to obtain multiple hyperparameter combinations. Multiple iterative evaluations are performed on the multiple hyperparameter combinations to obtain multiple performance results of each hyperparameter combination. Any iterative evaluation comprises obtaining at least one performance result of each hyperparameter combination, and if a hyperparameter combination meets a first preset condition, re-evaluating the hyperparameter combination to obtain a re-evaluated performance result of the hyperparameter combination. An optimal hyperparameter combination is determined. If the optimal hyperparameter combination does not meet a second preset condition, a preset model is updated, based on the multiple performance results of each hyperparameter combination, for next sampling. Or if the optimal hyperparameter combination meets a second preset condition, the optimal hyperparameter combination is used as a hyperparameter combination of a neural network.