Model Parameter Tuning Using Bounded Loss and Posterior Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing model parameter optimization methods face challenges in achieving high accuracy due to the disproportionate impact of prior data, leading to a concentration of the posterior distribution near optimal prior parameters, thereby neglecting the influence of posterior data.
Innovation Solution
A method that traverses reference values based on preset model evaluation information and data pairs to determine target values that meet specific loss and accuracy bounds, ensuring the optimized parameter considers both prior and posterior data, thereby updating the model to improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If sampling-based Bayesian model optimization is used to obtain posterior distribution based on prior distribution and loss values, then the optimization process can be automated, but the posterior distribution becomes exponentially concentrated near optimal prior parameters causing low model accuracy
Solution Approach 1:
The patent segments the parameter optimization process into discrete traversal steps, evaluating multiple reference values separately rather than relying on a single Bayesian posterior distribution. This segmentation allows the system to explore the parameter space more broadly and avoid premature convergence to a concentrated distribution.
Solution Approach 2:
The patent changes the optimization approach by introducing traversal of reference values with bounded loss functions, fundamentally altering how parameters are optimized. Instead of direct Bayesian updating that concentrates distribution, the system traverses reference values and selects those satisfying loss bounds, thereby maintaining diversity in the parameter distribution.
2Quantity of substance
If the amount of prior data is increased to improve training, then the prior distribution becomes more concentrated near optimal parameters, but the impact of posterior data is reduced leading to low model accuracy
Solution Approach 1:
The patent applies partial action by using only the necessary portion of prior data through bounded loss evaluation rather than fully relying on large prior datasets. The traversal mechanism evaluates reference values against loss bounds, allowing the system to achieve good performance without being dominated by excessive prior data.
Solution Approach 2:
The patent introduces feedback through the bounded loss function evaluation during traversal. The system continuously checks whether reference values satisfy the loss bounds and adjusts the selection of target values accordingly, creating a feedback loop that balances prior and posterior data influences.
3Measurement precision
If a complex hyperparameter is introduced to balance prior data and posterior data, then the model can achieve better accuracy, but the system complexity increases
Solution Approach 1:
The patent extracts the balancing mechanism from the traditional Bayesian framework by removing the need for complex hyperparameters. Instead of using hyperparameters to balance prior and posterior data, the system extracts the essential function through bounded loss evaluation and traversal, achieving simplicity while maintaining accuracy.
Solution Approach 2:
The patent introduces an intermediary mechanism—the bounded loss function and traversal process—that mediates between prior and posterior data without requiring complex hyperparameters. This intermediary evaluates reference values and selects target values that satisfy the loss bounds, effectively balancing the influence of different data sources.
Data Source
Figure 1
Figure 2a
Figure 2b
AI summary
Embodiments of this application provide a model parameter optimization method, a related system, and a storage medium. The method includes: obtaining a plurality of reference values of a parameter of a first model (301); traversing the plurality of reference values based on preset model evaluation information, the first model, and a prior data pair of the first model, to obtain at least one target value of the parameter (302), where a loss value that is of a model on the prior data pair and that corresponds to the target value is not greater than an upper bound of a loss value of a model on the prior data pair in the preset model evaluation information, and/or a test accuracy value that is of the model on the prior data pair and that corresponds to the target value is not less than a lower bound of a test accuracy value of the model on the prior data pair; determining an optimized value of the parameter based on the at least one target value, the prior data pair of the first model, and a posterior data pair of the first model (303); and updating the parameter of the first model based on the optimized value (304). In this way, model accuracy can be improved.