Apparatus for solving optimization problems, method for solving optimization problems, and program

Abstract

This optimization problem solution device is for performing calculation using an alternating direction multiplier method in which a variable update formula is represented by a linear equation comprising a coefficient vector and a coefficient matrix which includes an update parameter. The optimization problem solution device comprises: a matrix decomposition unit which decomposes, by using for the update parameter a step-type increasing sequence where the value only increases at specific numbers, the coefficient matrix into a lower triangular matrix and an upper triangular matrix only when the value of the sequence has increased at the specific numbers; and a variable update unit which obtains a solution to the linear equation by using the lower triangular matrix, the upper triangular matrix, and the coefficient vector for each of the numbers, and updates the value of the variable.

Description

[Technical Field]

[0001] The present invention relates to an optimization problem solving device, an optimization problem solving method, and a program. [Background technology]

[0002] The Alternating Direction Method of Multiplier (ADMM) is an iterative calculation method that can solve a wide class of optimization problems. Originally, ADMM was devised as a solver for solving convex problems. In recent years, due to progress in research on convergence in non-convex problems (Non-Patent Document 1), ADMM has been used in a wide range of fields, including graph simplification (Patent Document 1) and edge-preserving image smoothing (Non-Patent Document 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6810003 [Non-patent literature]

[0004] [Non-Patent Document 1] Y. Wang, W. Yin, and J. Zeng, “Global Convergence of ADMM in Nonconvex Nonsmooth Optimization,” J. Sci. Comput., vol. 78, no. 1, pp. 29-63, Jan. 2019. [Non-patent document 2] Takamoto Sasaki, Yukihiro Bando, and Masaki Kitahara, "Edge-preserving smoothing with simultaneous control of gradient and edge based on cusp total variation regularization," Proceedings of the 36th Signal Processing Symposium, IEICE, 2021, pp. 181-186. Summary of the Invention [Problem to be solved by the invention]

[0005] However, depending on how the optimization problem is set up, it may become necessary to repeatedly solve a huge linear equation during the iterative update of the ADMM, which becomes a bottleneck in calculation time. Non-Patent Document 2 uses a method that uses the fast Fourier transform (FFT) to quickly solve linear equations. However, this method can only be used when the input matrix of the optimization problem can be diagonalized by FFT. Therefore, for a general input matrix, it is not possible to quickly solve linear equations in the ADMM. Given the above situation, there is a need for a method to quickly solve the linear equations that are repeatedly executed in ADMM.

[0006] The present invention was made based on the above-mentioned recognition of the problem, and aims to provide an optimization problem solving device, an optimization problem solving method, and a program that can reduce the amount of calculation required to solve linear equations in the ADMM (Alternating Direction Method of Multipliers) and solve optimization problems more quickly. [Means for solving the problem]

[0007] One aspect of the present invention is an optimization problem solving device that calculates an alternating direction multiplier method in which an update formula for a variable is represented by a linear equation consisting of a coefficient matrix and a coefficient vector including update parameters, the optimization problem solving device including: a matrix decomposition unit that uses a step-type increase sequence, which is a sequence whose value increases only at specific numbers, as the update parameters, and decomposes the coefficient matrix into a lower triangular matrix and an upper triangular matrix only when the value of the sequence increases at the specific numbers; and a variable update unit that uses the lower triangular matrix, the upper triangular matrix, and the coefficient vector for each of the numbers to find a solution to the linear equation and update the values ​​of the variables.

[0008] One aspect of the present invention is a method for solving an optimization problem that calculates an alternating direction multiplier method in which an update formula for a variable is expressed as a linear equation consisting of a coefficient matrix and a coefficient vector including update parameters, the method including: a matrix decomposition step that uses a step-type increase sequence, which is a sequence whose value increases only at specific numbers, as the update parameters, and decomposes the coefficient matrix into a lower triangular matrix and an upper triangular matrix only when the value of the sequence increases at the specific numbers; and a variable update step that uses the lower triangular matrix, the upper triangular matrix, and the coefficient vector for each of the numbers to find a solution to the linear equation and update the values ​​of the variables.

[0009] One aspect of the present invention is a program for causing a computer to function as an optimization problem solving device for calculating an alternating direction multiplier method in which an update formula for a variable is expressed as a linear equation consisting of a coefficient matrix and a coefficient vector including update parameters, the program causing the computer to function as: a matrix decomposition unit that uses a step-type increase sequence, which is a sequence whose value increases only at specific numbers, as the update parameter, and decomposes the coefficient matrix into a lower triangular matrix and an upper triangular matrix only when the value of the sequence increases at the specific number; and a variable update unit that uses the lower triangular matrix, the upper triangular matrix, and the coefficient vector for each number to find a solution to the linear equation and update the values ​​of the variables. [Effects of the Invention]

[0010] According to the present invention, the amount of calculation required to solve linear equations in the ADMM (Alternating Direction Method of Multipliers) can be reduced, and optimization problems can be solved faster. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram for explaining an embodiment of the present invention, and is a graph showing an example of the relationship between a monotonically increasing sequence γk and a step-type increasing sequence γk step. [Figure 2] FIG. 1 is a schematic diagram showing Algorithm 1 (a method for solving an optimization problem). [Figure 3] FIG. 1 is a block diagram showing an example of a schematic functional configuration of an optimization problem solving device. [Figure 4] 10 is a flowchart illustrating an example of the flow of processing by an optimization problem solving device. DETAILED DESCRIPTION OF THE INVENTION

[0012] Next, an embodiment of the present invention will be described with reference to the drawings. First, a conventional ADMM (Alternating Direction Method of Multiplier) will be described, and then an ADMM according to the present invention (the present embodiment) will be described. After that, the specific functional configuration of an optimization problem solving device 1 according to the present embodiment will be described.

[0013] [About the conventional ADMM] The optimization problem that can be solved by ADMM is in the form of the following equation (1).

number

[0014] In contrast, the ADMM is expressed by the following update equations (2.1), (2.2), and (2.3).

number

[0015] By repeatedly executing the above update formula with the number k=0,1,..., the vector u (k) ,v (k) approaches the solution of the optimization problem. Here, the vector w (k) is an auxiliary variable. γ k is an update parameter in ADMM, which is set in advance by the user. When the optimization problem is non-convex, it has been said that it is good to set the update parameter to a monotonically increasing sequence (for example, Non-Patent Document 3).

[0016] [Non-patent document 3] S. Ono, “L_0 Gradient Projection,” IEEE Trans. Image Process., vol. 26, no. 4, pp. 1554-1564, 2017.

[0017] The most commonly used form of ADMM is when H(u) is a downward convex quadratic function. In this case, H(u) is

number

[0018] In this case, the first update equation (2.1) of the ADMM is

number

[0019] Since the objective function on the right-hand side is a downward convex quadratic function with respect to u, we can obtain a linear equation for the optimal solution by differentiating the objective function. The linear equation for equation (4) above is as follows:

number

[0020] Equation (5) is expressed as follows: the coefficient matrix and the coefficient vector are

number

[0021] [About the ADMM of the present invention] The ADMM of the present invention will be described, focusing on the differences from the conventional ADMM described above. In the present invention, the update parameter γ k From a monotonically increasing sequence to a step-type increasing sequence γ k step A step-type increasing sequence is a sequence in which the value increases only at certain specific numbers k1, k2, k3, ..., specifically,

number

[0022] A typical method for generating such a sequence is the monotonically increasing sequence γ k Prepare the following:

number

[0023] The above step-type increasing sequence is updated with the parameter γ k step Then, the coefficient matrix

number

number

[0024] Such matrix decomposition can be done in O(n 3 ) can be obtained with a computational complexity of γ k step While is constant, L k ,U k is also constant, the matrix decomposition needs to be performed only for specific updates where k=k1, k2, k3, . . .

[0025] L of the above matrix decomposition k ,U k is a triangular matrix, so solving the linear equations takes O(n 2 ) can be calculated efficiently. (k+1) teeth,

number

[0026] For example, the step-type increasing sequence γ with an increasing period of T expressed by Equation (8) k step In the case of the ADMM of the present invention, the computational complexity of equation (11) corresponding to update equation (2.1) is O(n 2 )+(1 / T)O(n 3 ) This means that when T is large enough, it becomes O(n 2 ) is considered to be close to

[0027] FIG. 2 is a schematic diagram showing the ADMM algorithm of the present invention. This algorithm is written in pseudo-code. The algorithm will be explained below with reference to this diagram. In this algorithm, the inputs are an n-by-n semi-definite symmetric matrix P, an n-dimensional vector q, a scalar r, a function G, an m-by-n matrix A, an m-by-l matrix B, an m-dimensional vector b, and a step-type increasing sequence γ k The output is the solution (u, v) of the optimization problem in the following equation (12).

number

[0028] In the following explanation, we will refer to the line numbers on the left side of the figure. In the first line, the variable v (0) zero vector, variable w (0) a zero vector, variable k is 0, γ -1 Substitute -Inf (negative Infinity) for .

[0029] Lines 2 to 11 indicate that the process is repeated between these two lines. This repeated process ends when the end condition is met in line 10.

[0030] Lines 3 to 5 show the conditional branch. k ≠γ k-1 If (step-type increasing sequence γ k Line 4 is executed only if γ -1 Since is -Inf, line 4 is always executed when k=0. In the fourth row, the coefficient matrix

number

[0031] In line 6, the variable u (k+1) First, the coefficient vector

number

[0032] In line 7, the variable v (k+1) The solution v of the following minimization problem is substituted into v (k+1) Substitute into

number

[0033] In line 8, the variable w (k+1) is assigned to .

number

[0034] In line 9, an assignment is made to the variable k: the value of k+1 is assigned to k. Line 12 is executed when the iteration of lines 2 to 11 is completed. In line 12, the outputs u and v are assigned. u (k) The value of u is set to v (k) The algorithm ends by substituting the value of

[0035] [Optimization problem solving device] Next, an optimization problem solving device 1 according to this embodiment will be described. Next, FIG. 3 is a block diagram showing a schematic functional configuration of the optimization problem solving device 1 according to this embodiment. As shown in FIG. 3, the optimization problem solving device 1 includes an initial setting unit 11, a matrix decomposition unit 12, and a variable update unit 13. Each unit also includes a storage unit for storing data as needed. This storage unit is realized using storage means such as a semiconductor memory or a magnetic hard disk drive. The functions of each unit may also be realized by a computer and a program.

[0036] The optimization problem solving device 1 receives as input an n-by-n semi-positive definite symmetric matrix P, an n-dimensional vector q, a scalar r, a function G, an m-by-n matrix A, an m-by-l matrix B, an m-dimensional vector b, and a step-type increasing sequence γ k (k=0,1,…) and solve the optimization problem

number

[0037] The initial setting unit 11 sets the vector variable v (0) , vector variable w (0) , variable k, variable γ -1 Set the initial values ​​to and output. For example, these values ​​are a zero vector, a zero vector, 0, and -Inf (negative Infinity), respectively.

[0038] The matrix decomposition unit 12 calculates P, A, γ k (k=0,1,…) as input and the coefficient matrix

number

[0039] The variable update unit 13 updates P, q, r, G, A, B, b, γ k (k=0,1,…),v (0) ,w (0) ,k,γ -1 ,L,U as input, and calculate u according to the following equations (19.1), (19.2), (19.3). (k+1) ,v (k+1) ,w (k+1) Then, when the termination condition is met, u (k) ,v (k) is output as u,v.

[0040]

number

[0041] Specifically, the variable update unit 13 (0) ,w (0) ,k=0 is the initial value. First, the variable update unit 13 updates γ k ≠γ k-1 If this holds, L and U are obtained from the matrix decomposition unit 12. Then, the variable update unit 13 calculates v (k),w (k) ,L,U,γ k etc., according to equations (19.1), (19.2), and (19.3), u (k+1) ,v (k+1) ,w (k+1) Then, the variable update unit 13 calculates u (k) ,v (k) ,w (k) The value of u (k+1) ,v (k+1) ,w (k+1) The variable update unit 13 updates the value of u when the termination condition is satisfied. (k) ,v (k) is output as u,v.

[0042] Next, a description will be given of the processing flow of the optimization problem solving device 1 according to this embodiment. Fig. 4 is a flowchart showing the processing flow of the optimization problem solving device 1 according to this embodiment.

[0043] First, the optimization problem solving device 1 receives as input an n-by-n semi-positive definite symmetric matrix P, an n-dimensional vector q, a scalar r, a function G, an m-by-n matrix A, an m-by-l matrix B, an m-dimensional vector b, and a step-type increasing sequence γ k (k=0, 1, . . . ) is accepted (step S101).

[0044] Next, the initial setting unit 11 sets the variable v (0) zero vector, variable w (0) to a zero vector, variable (number) k to 0, γ -1 is set to -Inf (negative infinity) (step S102).

[0045] Next, the matrix decomposition unit 12 calculates γ k ≠γ k-1 (Step-type increasing sequence γ k It is determined whether γ increases or not (step S103). -1 is set to -Inf, so that γ0 ≠ γ whenever k = 0. -1 holds true.

[0046] If the answer to step S103 is Yes, the matrix decomposition unit 12 decomposes the coefficient matrix

number

[0047] Next, the variable update unit 13 first calculates the coefficient vector

number

[0048] Next, the variable update unit 13 solves the minimization problem of the following equation (22) and updates the solution v with the variable v (k+1) (step S106).

number

[0049] Next, the variable update unit 13 calculates the value of the following equation (23) and obtains w (k+1) (step S107).

number

[0050] Next, the variable update unit 13 updates u (k) ,v (k) ,w (k) The value of u (k+1) ,v (k+1) ,w (k+1) (step S108). Next, the variable update unit 13 updates the value of the number k by k+1 (step S109).

[0051] Next, the variable update unit 13 determines whether or not a termination condition is met (step S110). The termination condition may be, for example, that the repetition has been performed a predetermined number of times, or that u (k) ,v (k) The fluctuations in have become smaller.

[0052] If the answer is No in step S110 (the termination condition is not met), the optimization problem solving device 1 returns the control to step S103. If the answer is Yes in step S110 (the termination condition is met), the optimization problem solving device 1 (variable update unit 13) updates (u, v)=(u (k) ,v (k) ) is output as a result (step S111). This concludes the description of the processing flow of the optimization problem solving device 1 in FIG.

[0053] All or part of the functions of the optimization problem solving device 1 in each of the above-described embodiments may be implemented by a computer. In this case, a program for implementing these functions may be recorded on a computer-readable recording medium, and the program may be loaded into a computer system and executed by a CPU (controller). Note that the term "computer system" as used herein includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, CD-ROMs, DVD-ROMs, and USB memory, as well as storage devices such as hard disks built into a computer system. Furthermore, the term "computer-readable recording medium" may also include media that dynamically store programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or telephone lines, or media that store programs for a fixed period of time, such as volatile memory within a computer system that serves as a server or client. The program may also be a program that implements part of the above-described functions, or may be a program that can implement the above-described functions in combination with a program already stored in the computer system.

[0054] [effect] As described above, the optimization problem solving device according to this embodiment can reduce the amount of calculation required to solve linear equations in the alternating direction method of multipliers (ADMM), and therefore can execute a high-speed calculation algorithm for the alternating direction method of multipliers. Therefore, the optimization problem solving device can solve optimization problems that use the alternating direction method of multipliers more quickly.

[0055] More specifically, matrix decomposition is O(n 3) (cubed n, the size of the symmetric matrix P), but as mentioned above, the coefficient matrix is ​​a constant value in most cases (except when the value of the step-type increasing sequence increases), so the number of times matrix decomposition is performed can be reduced. Furthermore, solving linear equations for triangular matrices can be done in O(n 2 ) can be calculated efficiently. Therefore, the optimization problem solver can solve the problem efficiently with a computational complexity of O(n 2 ) can solve linear equations with a computational effort close to that of

[0056] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like are also included within the scope that does not deviate from the gist of the present invention. [Industrial Applicability]

[0057] The present invention is applicable to techniques for solving optimization problems using the Alternating Direction Method of Multipliers (ADMM), and is also applicable to a wide range of techniques such as graph simplification and edge-preserving image smoothing that can be formulated as an optimization problem using alternating direction multipliers. [Explanation of symbols]

[0058] 1...Optimization problem solving device 11...Initial setting section 12...matrix decomposition part 13. Variable update section

Claims

1. An optimization problem solving device for calculating an alternating direction multiplier method in which an update formula for a variable is expressed as a linear equation consisting of a coefficient matrix and a coefficient vector including update parameters, a matrix decomposition unit that uses a step-type increasing sequence, which is a sequence whose value increases only at a specific number, as the update parameter, and decomposes the coefficient matrix into a lower triangular matrix and an upper triangular matrix only when the value of the sequence increases at the specific number; a variable update unit that finds a solution to the linear equation using the lower triangular matrix, the upper triangular matrix, and the coefficient vector for each of the numbers, and updates the values ​​of the variables; An optimization problem solving device comprising:

2. When the value of the sequence is not the specific number that increases, the variable update unit uses the same lower triangular matrix and upper triangular matrix as the lower triangular matrix and upper triangular matrix used for the previous number.

2. The optimization problem solving device according to claim 1.

3. The step-type increasing sequence increases in value at a fixed number cycle.

2. The optimization problem solving device according to claim 1.

4. The coefficient matrix is ​​a semi-positive definite symmetric matrix with n rows and n columns, P, an m-row and n-column matrix, A, the number k=0, 1, . . . , and the step-type increasing sequence γ ｋ ｓｔｅｐ When [Equation 1] is expressed as 2. The optimization problem solving device according to claim 1.

5. The linear equation is expressed by A, where A is an m-row, n-column matrix, B is an m-row, l-column matrix, q is an n-dimensional vector, b is an m-dimensional vector, k=0, 1, . . . , and γ is a step-type increasing sequence. ｋ ｓｔｅｐ , the lower triangular matrix is ​​L ｋ , the upper triangular matrix is ​​U ｋ , the variables updated by other update equations are v （ｋ） , w （ｋ） , the desired variable is u （ｋ＋１） When [Equation 2] is expressed as 2. The optimization problem solving device according to claim 1.

6. 1. A method for solving an optimization problem in which a computer calculates an alternating direction multiplier method in which an update formula for a variable is expressed as a linear equation consisting of a coefficient matrix and a coefficient vector including update parameters, a matrix decomposition step in which a step-type increasing sequence, which is a sequence whose value increases only at a specific number, is used as the update parameter, and the coefficient matrix is ​​decomposed into a lower triangular matrix and an upper triangular matrix only when the value of the sequence increases at the specific number; a variable updating step of finding a solution to the linear equation using the lower triangular matrix, the upper triangular matrix, and the coefficient vector for each of the numbers, and updating the values ​​of the variables; A method for solving optimization problems, including

7. A program for causing a computer to function as an optimization problem solving device for calculating an alternating direction multiplier method in which an update formula for a variable is expressed by a linear equation consisting of a coefficient matrix and a coefficient vector including update parameters, the program comprising: a matrix decomposition unit that uses a step-type increasing sequence, which is a sequence whose value increases only at a specific number, as the update parameter, and decomposes the coefficient matrix into a lower triangular matrix and an upper triangular matrix only when the value of the sequence increases at the specific number; a variable update unit that finds a solution to the linear equation for each of the numbers using the lower triangular matrix, the upper triangular matrix, and the coefficient vector, and updates the values ​​of the variables; A program to function as a

Images