Optimization problem processing method and device for machine learning

Abstract

The invention discloses an optimization problem processing method and device for machine learning. The method comprises the steps: determining an objective function and a constraint condition according to a to-be-solved problem, and continuing a next step in response to the fact that the objective function is a non-differentiable convex function; updating the current solution by using a projection subgradient method based on the number of iterations and the constraint condition, and substituting the current solution into the target function to obtain a current function value; in response to the fact that the current function value does not meet the convergence condition and the number of iterations does not exceed the preset upper limit, accumulating the number of iterations and re-executing the previous step; and determining a solution corresponding to an optimal value in the current function values as an optimal solution of the to-be-solved problem in response to the condition that the current function values meet the convergence condition or the number of iterations exceeds a preset upper limit. According to the method, the optimal solution problem of the non-differentiable function of machine learning can be solved, and the convergence speed is very high.

Description

technical field [0001] The present invention relates to the field of optimization, more specifically, a method and device for processing optimization problems of machine learning. Background technique [0002] In the field of machine learning, optimization theory is the most important. The main reason is that machine learning often abstracts the problem into an optimization problem, such as maximizing rewards, minimizing losses caused by classification errors, or maximizing likelihood, etc., and how to solve these problems requires corresponding optimization algorithms. The current mainstream algorithm is SGD, and many variants of SGD, including Momentum, AdaGrad, RMSProp, Adam, etc. What these algorithms have in common is that they all need to calculate the gradient value of the objective function, but in practical applications, the gradient does not necessarily exist, and it is easy for the gradient to disappear during the training process. Therefore, how to solve the pr...

AI Technical Summary

Problems solved by technology

[0004] There is currently no effective solution to the problem in the prior art that the gradient of the objective function disappears and affects the training process of machine learning

Images