Machine Learning Program Using Continuous Relaxation and Dynamic Penalties
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing combinatorial optimization methods, such as those using Ising machines and continuous relaxation simulated annealing, struggle to efficiently find multiple solutions due to issues with penalty coefficients and local solutions, leading to inefficiencies in obtaining various solutions.
Innovation Solution
A machine learning program that employs a continuous relaxation method with simultaneous optimization of multiple penalty coefficients and Bayesian optimization of the penalty term, using a surrogate function to train the penalty term and adjust the penalty coefficient dynamically, allowing for the simultaneous generation of multiple solutions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional combinatorial optimization methods (Ising machines, continuous relaxation simulated annealing) are used, then optimization can be performed, but multiple solutions cannot be efficiently obtained due to penalty coefficient issues and local solution trapping
Solution Approach 1:
The patent segments the optimization process by introducing multiple penalty coefficients (α1, α2, ..., αk) that divide the search space into different regions. Each penalty coefficient corresponds to a specific constraint violation type, allowing the system to explore multiple solution regions simultaneously and efficiently find multiple distinct solutions without getting trapped in local optima.
Solution Approach 2:
The patent dynamically changes penalty coefficients during the optimization process. By adjusting penalty coefficients based on constraint violation patterns, the system can transition between different solution regions and discover multiple solutions. This parameter adaptation allows efficient exploration of the solution space while maintaining reliability in finding diverse solutions.
2Adaptability or versatility
If penalty coefficients are fixed in traditional methods, then the optimization process is simple, but the system cannot adapt to find various solutions efficiently
Solution Approach 1:
The patent introduces dynamic penalty coefficients that adapt during the optimization process. Instead of fixed penalty coefficients, the system continuously adjusts α1, α2, ..., αk based on the current state of constraint violations and progress toward solutions. This dynamic adaptation enables the system to handle various optimization problems effectively while maintaining manageable complexity through systematic update rules.
Solution Approach 2:
The patent implements feedback mechanisms where penalty coefficients are adjusted based on information from previous optimization steps. The system monitors constraint violations and solution quality, then uses this feedback to modify penalty coefficients for subsequent iterations. This feedback loop enables adaptability in finding various solutions while keeping the complexity controlled through structured feedback processing.
3Ease of manufacture
If continuous relaxation is applied to discrete variables, then the optimization problem becomes continuous and easier to solve, but obtaining discrete solutions requires additional processing
Solution Approach 1:
The patent performs preliminary rounding of continuous variables to discrete values during the optimization process itself, rather than as a separate post-processing step. By integrating the rounding operation into the continuous optimization framework and using penalty coefficients to guide the rounding decisions, the system efficiently obtains discrete solutions that satisfy constraints while maintaining the computational advantages of continuous optimization.
Data Source
AI summary
A non-transitory computer-readable recording medium storing a machine learning program for causing a computer to execute a process including training a machine learning model by machine learning that uses a cost function in which each element of a matrix obtained by relaxing a discrete variable to be optimized to a continuous matrix becomes a discrete optimization problem as a cost function in a search process that performs a search by adopting continuous relaxation into the discrete optimization problem.