Solution apparatus, solution method, and program

Abstract

To suppress the occurrence of perturbation disappearance and amplitude underestimation.SOLUTION: A control unit configured to execute a solution process of obtaining a solution of an underdetermined inverse problem by using regularization in which an inner product of an absolute value vector and an action result vector is a regularization term, the absolute value vector being a vector obtained based on an image vector and satisfying a condition that a value of an element that is an m-th (m is an integer of 1 or more) element number from a smallest element number indicates an m-th largest absolute value among values of elements of the image vector, the image vector is a result of linear transformation of a vector having optimization variables as elements, and the action result vector is a result of action of a predetermined diagonal matrix on the absolute value vector from the left, the predetermined diagonal matrix having non-negative values of diagonal components and satisfying a condition that a value indicated by a diagonal component with a smaller row number is equal to or smaller than a value indicated by a diagonal component with a larger row number.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a solution-finding device, a solution-finding method, and a program. [Background technology]

[0002] Underdetermined inverse problems are a technology required in many areas, such as sensing and computational imaging. As a result, research into them is active, and various regularization techniques and solution algorithms have been proposed. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] T. Sasaki, Y. Bandoh, and M. Kitahara, “Sparse Regularization Based on Reverse Ordered Weighted L1-Norm and Its Application to Edge-Preserving Smoothing,” in ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, Apr. 2024, pp. 9531-9535. Summary of the Invention [Problem to be solved by the invention]

[0004] Solutions to such inverse problems can be obtained through optimization, but optimization is performed on a regularized function representing the inverse problem to improve estimation accuracy and generalization performance. However, regularization can cause perturbation loss and amplitude underestimation. Perturbation loss and amplitude underestimation occur particularly in the widely used sparse regularization. Note that perturbation loss refers to the phenomenon in which the magnitude of the estimated signal, which is an estimated signal when the magnitude of the original signal is small, becomes almost zero, and amplitude underestimation refers to the phenomenon in which the difference between the original signal and the estimated signal is large.

[0005] Therefore, Non-Patent Document 1 proposed ROWL (Reverse Ordered Weighted L1-Norm) as a regularization technique. ROWL is a regularization technique that suppresses the occurrence of perturbation disappearance and amplitude underestimation more than sparse regularization. However, regularization techniques are not applicable to all optimizations. The condition for applicability here is that, even if applied, it does not make it impossible to execute calculations according to the algorithm, and a solution is obtained that is guaranteed to suppress the occurrence of perturbation disappearance and amplitude underestimation.

[0006] In view of the above circumstances, an object of the present invention is to provide a technique for suppressing the occurrence of perturbation loss and amplitude underestimation. [Means for solving the problem]

[0007] One aspect of the present invention is a solution solving device comprising: a control unit that executes a solution-finding process to obtain a solution to an underdetermined inverse problem using regularization in which the inner product of an absolute value vector and an action result vector is used as a regularization term; the absolute value vector is a vector obtained based on an image vector, and satisfies a condition that an element value having an m-th smallest element number (m is an integer equal to or greater than 1) has an absolute value that is the m-th largest among the element values ​​of the image vector; the image vector is a result of a linear transformation of a vector whose elements are optimization variables; and the action result vector is a result of applying, from the left of the absolute value vector, a predetermined diagonal matrix whose diagonal components are nonnegative, and which satisfies a condition that a diagonal component with a smaller row number has an equal to or smaller than a diagonal component with a larger row number.

[0008] One aspect of the present invention is a solution-finding method executed by a solution-finding device, the method comprising: a control unit that executes a solution-finding process to obtain a solution to an underdetermined inverse problem using regularization in which the inner product of an absolute value vector and an action result vector is used as a regularization term; the absolute value vector is a vector obtained based on an image vector, and satisfies a condition that an element value having an m-th smallest element number (m is an integer equal to or greater than 1) has an absolute value that is the m-th largest among the element values ​​of the image vector; the image vector is a result of a linear transformation of a vector whose elements are optimization variables; and the action result vector is a result of applying, from the left of the absolute value vector, a predetermined diagonal matrix whose diagonal elements have nonnegative values, and which satisfies a condition that a diagonal element with a smaller row number has an equal to or smaller than a diagonal element with a larger row number; the solution-finding method comprises a control step of executing the solution-finding process.

[0009] One aspect of the present invention is a program for causing a computer to function as the above-described solution-finding device. [Effects of the Invention]

[0010] The present invention makes it possible to suppress the occurrence of perturbation loss and amplitude underestimation. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is an explanatory diagram illustrating a solution-finding apparatus according to an embodiment. [Figure 2] FIG. 2 is a diagram showing an example of the hardware configuration of a solution solving apparatus according to an embodiment. [Figure 3] 3 is a flowchart showing an example of a flow of processing executed by the solution finding device in the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] (Embodiment) 1 is an explanatory diagram illustrating a solution-finding device 1 according to an embodiment. The solution-finding device 1 includes a control unit 11 including a processor 91, such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an NPU (Neural Network Processing Unit), and a memory 92, which are connected via a bus.

[0013] The control unit 11 executes, for example, a solution-finding process. The solution-finding process is a process for obtaining a solution to a problem to be solved (hereinafter referred to as "the problem to be solved") using regularization with the inner product of an absolute value vector and an action result vector as a regularization term. The solution to the problem to be solved is a solution obtained by this DC optimization. Specifically, the problem to be solved is an underdetermined inverse problem.

[0014] An absolute vector is a vector obtained based on an image vector. More specifically, it is a vector that satisfies the condition that the value of the element with the smallest element number m (m is an integer equal to or greater than 1) is the mth largest absolute value among the element values ​​of the image vector.

[0015] The image vector is the result of a linear transformation (hereinafter referred to as "variable vector transformation") of a vector whose elements are optimization variables (hereinafter referred to as "variable vector"). The variable vector transformation may be expressed, for example, by a square matrix or a non-square matrix. The variable vector transformation may be an identity transformation or a transformation different from the identity transformation.

[0016] The action result vector is the result of applying the action diagonal matrix to the absolute value vector from the left. The action diagonal matrix is ​​a predetermined diagonal matrix whose diagonal elements have non-negative values ​​and whose diagonal elements with smaller row numbers have values ​​less than or equal to the diagonal elements with larger row numbers.

[0017] <Explanation using the formula for the regularization term in the solution process> The regularization term in the solution-finding process will be explained using a mathematical formula. The regularization term in the solution-finding process is, for example, Ψ expressed by the following formula (1): w (Bx). Below, Ψ w (Bx) is called the RSSS function. Note that N is a predetermined integer equal to or greater than 1, and n is an integer equal to or greater than 1 and equal to or less than N.

[0018]

number

[0019]

number

[0020] The right side of equation (1) may be multiplied by a predetermined coefficient such as 1 / 2. n is the weight of a given scalar that satisfies the condition of equation (2). Therefore, the weight of a given scalar is the weight w nWhen the regularization term is set to q times (q is a predetermined real number) and the right side of equation (1) is multiplied by the coefficient 1 / q, the regularization term is equal to the regularization term of equation (1). Therefore, the presence or absence of a coefficient does not affect the results of the solution process. For simplicity of explanation, the following explanation will be given using equation (1) with a coefficient of 1 as an example.

[0021] x is a variable vector. Therefore, each element of the variable vector x is an optimization variable. The variable vector x can be expressed by the following equation (3) by specifying each element.

[0022]

number

[0023] In equation (1), B is a matrix representing a variable vector transformation (hereinafter referred to as a "variable vector transformation matrix"). Therefore, Bx is an example of an image vector.

[0024] |Bx| in Eq. (1) ↓ is an absolute value vector obtained based on the image vector Bx.

[0025] << Absolute value vector |Bx| ↓ Example>> Just to be sure, the absolute value vector |Bx| ↓ When each element of the image vector Bx satisfies the condition expressed by the following equation (4), for example, the vector |Bx| ↓ is expressed by the following formula (5). In formula (4), Bx n means the n-th element of the image vector Bx. Therefore, |Bx n | means the absolute value of the n-th element of the image vector Bx.

[0026]

number

[0027]

number

[0028] [|Bx| ↓ ] n is the absolute value vector |Bx| ↓ Therefore, for example, in the example of formula (5), [|Bx| ↓ ]1 is |Bx N |Bx| ↓ ] n 2 is [|Bx| ↓ ] n means the square of .

[0029] Thus, the right side of equation (1) is the absolute value vector |Bx| ↓ Therefore, the right-hand side of equation (1) is transformed into the right-hand side of equation (6) below. In other words, the right-hand side of equation (1) satisfies the relationship of equation (6) below.

[0030]

number

[0031] Note that diag(w) is an N × N matrix, and the element in the nth row and nth column is w n Therefore, diag(w) is an example of a diagonal matrix that satisfies the condition that the values ​​of the diagonal elements are non-negative and the values ​​indicated by the diagonal elements with smaller row numbers are less than or equal to the values ​​indicated by the diagonal elements with larger row numbers. In other words, diag(w) is an example of an action diagonal matrix.

[0032] The right-hand side of equation (6) is the diagonal matrix diag(w) of the absolute value vector |Bx| ↓ The result of applying from the left and the absolute value vector |Bx| ↓ The action diagonal matrix diag(w) is the absolute value vector |Bx| ↓ The result of acting on from the left is an example of an action result vector.

[0033] <Effects of solution-finding processing> When regularization is performed using the RSSS function, amplitude underestimation and perturbation loss are actually suppressed. This will be explained below. As can be seen from the definition of the RSSS function, the RSSS function assigns a smaller weight to the image vector as the value of the image vector increases. This is similar to ROWL described in Non-Patent Document 1. Therefore, regularization using the RSSS function suppresses amplitude underestimation, similar to ROWL described in Non-Patent Document 1.

[0034] Furthermore, as can be seen from the definition of the RSSS function, it is a piecewise quadratic function of the image vector. Therefore, the larger the image vector, the larger the slope of the RSSS function, and the smaller the image vector, the smaller the slope of the RSSS function. If the original signal is small, the estimated signal is also small, so perturbation loss occurs. However, the smaller the image vector (i.e., the smaller the original signal), the smaller the change in the RSSS function, so there is a higher chance that the image vector will not become zero even if the original signal becomes small. Therefore, there is a higher chance that the variable vector will not become zero. Therefore, the RSSS function can suppress the occurrence of perturbation loss.

[0035] In this way, the solution process that incorporates the RSSS function as a regularization term to obtain a solution to the problem to be solved can suppress the occurrence of perturbation disappearance and amplitude underestimation.

[0036] <Example of hardware configuration> 2 is a diagram showing an example of the hardware configuration of the solution-finding device 1 in the embodiment. The solution-finding device 1 has a control unit 11 and executes a program. By executing the program, the solution-finding device 1 functions as a device having the control unit 11, an interface unit 12, and a storage unit 13.

[0037] More specifically, the processor 91 reads out a program stored in the storage unit 13 and stores the read out program in the memory 92. When the processor 91 executes the program stored in the memory 92, the solution-finding device 1 functions as a device including the control unit 11, the interface unit 12, and the storage unit 13.

[0038] The control unit 11 controls the operation of each functional unit included in the solution-finding device 1. The control unit 11 executes, for example, a solution-finding process. The control unit 11 acquires, for example, information stored in the memory unit 13. Specifically, the process of acquiring information stored in the memory unit 13 is reading.

[0039] The interface unit 12 includes a communication interface for connecting the solution-finding device 1 to an external device. The interface unit 12 communicates with the external device via wire or wirelessly.

[0040] The external device is, for example, a device that transmits information used for execution by the control unit 11. In such a case, the interface unit 12 acquires the information used for execution by the control unit 11 from the device that transmits the information used for execution by the control unit 11. The device that transmits the information used for execution by the control unit 11 is, for example, a device that transmits information indicating a function that represents the problem to be solved (hereinafter referred to as "problem information to be solved"). Since the problem information to be solved is information that indicates a function that represents the problem to be solved, the problem information to be solved can also be said to be information that indicates the problem to be solved in the form of a function.

[0041] The interface unit 12 may be configured to include input devices such as a mouse, keyboard, touch panel, etc. The interface unit 12 may be configured as an interface that connects these input devices to the solution-finding device 1. In this way, the input device of the interface unit 12 accepts input of various information to the solution-finding device 1 via wired or wireless connections. Note that the information does not necessarily have to be input to the communication interface of the interface unit 12, but may also be input to the input device of the interface unit 12. Therefore, information about the problem to be solved may be input to the input device of the interface unit 12, for example.

[0042] The interface unit 12 outputs, for example, various types of information. The interface unit 12 includes a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, or an organic EL (Electro-Luminescence) display, and a speaker. The interface unit 12 may be configured as an interface that connects these display devices or speakers to the solution-finding device 1. Therefore, the display device and speaker included in the interface unit 12 output, for example, information acquired by a communication interface of the interface unit 12 or information input to an input device of the interface unit 12 as an image or sound.

[0043] The storage unit 13 is configured using a computer-readable storage medium device (non-transitory computer-readable recording medium) such as a magnetic hard disk device or a semiconductor storage device. The storage unit 13 stores various information related to the solution-finding device 1. The storage unit 13 stores various information generated by the operation of the control unit 11, for example. The storage unit 13 stores information used in the solution-finding process and information generated by the execution of the solution-finding process, for example. Therefore, the storage unit 13 may store information indicating the operating diagonal matrix in advance, for example. The storage unit 13 may exist on a cloud, for example.

[0044] 3 is a flowchart showing an example of the flow of processing executed by the solution-finding device 1 in the embodiment. The control unit 11 acquires solution-finding target problem information (step S101). Next, the control unit 11 executes solution-finding processing (step S102). By executing the solution-finding processing, a solution to the solution-finding target problem, which is indicated in the form of a function by the solution-finding target problem information obtained in step S101, is obtained.

[0045] The solution-finding device 1 configured in this way executes the solution-finding process, which makes it possible to prevent the occurrence of perturbation disappearance and amplitude underestimation, as described in <Effects of the solution-finding process>.

[0046] (Variation) By the way, the RSSS function Ψw (Bx) satisfies the relationship of the following equation (7). Hereinafter, the theorem that shows the relationship of equation (7) will be called the Sasaki-Bando-Kitahara-Ono theorem.

[0047]

number

[0048]

number

[0049]

number

[0050] <<Proof of the Sasaki-Bando-Kitahara-Ono Theorem>> Prove the Sasaki-Bando-Kitahara-Ono theorem.

[0051] From equations (1) and (6), the following equation (10) is established.

[0052]

number

[0053] From equation (8), the function Ψ ω (Bx) is the RSSS function Ψ w Since w in (Bx) is replaced by ω, the following equation (11) holds.

[0054]

number

[0055] Therefore, by considering the relationship between each component of vector w and each component of vector ω in equation (9), we obtain equation (7).

[0056] The first term on the right side of equation (7) is a convex function. Also, the function Ψ ω(Bx) is also a convex function. This is because ω is monotonically non-increasing.

[0057] <Relationship with DC Optimization> By the way, one of the optimization techniques for obtaining the solution of an ill-determined inverse problem is DC optimization (Difference of Convex function), which minimizes the difference between two convex functions. Therefore, in the solution process, obtaining the solution by DC optimization may be performed. In this case, the solution of the problem to be solved obtained in the solution process is the result of DC optimization.

[0058] Explain specifically what kind of convex functions the two convex functions in the DC optimization executed in the solution process are. One of the two convex functions is the sum of the L2 convex function and the function representing the problem to be solved. The L2 convex function is the product of the maximum value among the values of the elements of the action diagonal matrix and the L2 norm of the action result vector. The first term on the right side of Equation (7) is an example of the L2 convex function.

[0059] The other of the two convex functions is the result of subtracting the RSSS function from the L2 convex function. Therefore, according to the Sasaki - Bando - Kitahara - Ono theorem, the function Ψ ω (Bx) is an example of the other of the two convex functions above. The function f(x) is a convex function representing the problem to be solved.

[0060] Therefore, the DC function in the DC optimization executed in the solution process is represented by the following Equation (12).

[0061]

Number

[0062] <<Relationship between DC Optimization and RSSS Function>> Equation (12) can be transformed into the following Equation (13) by using the relationship shown by the Sasaki - Bando - Kitahara - Ono theorem (that is, Equation (7)).

[0063]

Number

[0064] Therefore, the result of DC optimization expressed by Equation (12) is actually the solution of the target problem obtained under regularization by the RSSS function, and therefore, such DC optimization can suppress the occurrence of perturbation loss and amplitude underestimation.

[0065] <The significance of using the function of equation (12) as a DC function> By the way, why not use the function in equation (12) as the DC function, rather than using the function representing the problem to be solved minus the RSSS function? Let us explain this point. In fact, the RSSS function is a non-convex function because ω is monotonically non-decreasing. Therefore, if one of the two convex functions in the DC function is replaced with the RSSS function, it may become impossible to perform calculations according to the DC optimization algorithm.

[0066] On the other hand, in the case of the DC function of equation (12), the function in the parentheses on the right side of equation (12) is a convex function because it is the sum of the function f(x), which is a convex function, and an L2 convex function. ω As mentioned above, (Bx) is a convex function. Therefore, the DC function in equation (12) has the form of the difference between a convex function and another convex function. Therefore, it is possible to perform calculations according to the DC optimization algorithm. Therefore, the DC function expressed in equation (12) is used to perform solution processing using DC optimization.

[0067] There are various algorithms for performing DC optimization, but among them, an algorithm that is guaranteed to converge to a stationary point may be used as the DC optimization algorithm. The solution process for performing DC optimization using such an algorithm not only further suppresses the occurrence of perturbation loss and amplitude underestimation, but also obtains a solution that is guaranteed to converge to a stationary point.

[0068] The algorithm used in DC optimization that is guaranteed to converge to a stationary point may be, for example, the general double-proximal gradient algorithm. Specifically, in the solution process for DC optimization using the general double-proximal gradient algorithm, alternating optimization of the following equations (14) and (15) is performed. This alternating optimization may use, for example, the following equations (17) to (22). Note that equation (17) is an equation obtained by Moreau decomposition. Note that γ1 is defined as a positive step size, and γ2 is defined as a positive step size.

[0069]

number

[0070]

number

[0071]

number

[0072]

number

[0073]

number

[0074] The right side of equation (16) can be solved by the technique described in Reference 1 below.

[0075] Reference 1: Frank MTA Busing “Monotone regression: A simple and fast O (n) PAVA implementation.” Journal of Statistical Software 102 (2022): 1-25.

[0076] Here we show the derivation of the proximity map of FSSS (Forward Sorted Sum of Squares).

[0077]

number

[0078]

number

[0079] Here, according to the Hardy-Littlewood-Polya rearrangement inequality, the following equation (21) holds.

[0080]

number

[0081] Therefore, the right side of equation (20) can be transformed as shown in equation (22) below.

[0082]

number

[0083] Here, if the technique described in Reference 2 below is used, Equation (22) can be further transformed to obtain Equation (23) below.

[0084] Reference 2: Nemeth, AB, and SZ Nemeth. “How to project onto the monotone nonnegative cone using pool adjacent violators type algorithms.” arXiv preprint arXiv:1201.2343 (2012).

[0085]

number

[0086] <Application example> An application example of the solution-finding process will be described. The solution-finding process may be used for, for example, a linear inverse problem of an image. Specifically, the solution-finding process for a linear inverse problem of an image is expressed by the following equation (24).

[0087]

number

[0088] Here, x represents the original image. Therefore, x is a variable vector. y represents the observed image, and A represents the observation matrix. Furthermore, D represents the adjacent difference matrix. D is an example of a variable vector transformation matrix. The third term in the parentheses on the right side of equation (24) is a convex function that can be nearby mapped, and specifically, is an indicator function for the following equation (27). In other words, the third term is a function obtained by rewriting the constraint expressed by the following equation (27) into an objective function.

[0089] When solving the optimization problem of equation (24) using the general double-proximal gradient algorithm, the following equations (25) and (26) are alternately optimized.

[0090]

number

[0091]

number

[0092]

number

[0093] The linear inverse problem of the image to which the solution process is applied may be, more specifically, for example, a super-resolution problem, a compressed sensing problem, or a colorization problem.

[0094] The solution-finding device 1 may be implemented using a plurality of information processing devices communicably connected via a network. In this case, the functional units of the solution-finding device 1 may be distributed and implemented among the plurality of information processing devices.

[0095] All or part of the functions of the solution-finding device 1 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. The program may be transmitted via a telecommunications line.

[0096] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention. [Explanation of symbols]

[0097] 1...solution-finding device, 11...control unit, 12...interface unit, 13...storage unit, 91...processor, 92...memory

Claims

1. a control unit that executes a solution process for obtaining a solution to an underdetermined inverse problem using regularization in which the inner product of an absolute value vector and an action result vector is used as a regularization term; Equipped with the absolute value vector is a vector obtained based on an image vector, and satisfies the condition that the value of the element having the mth smallest element number (m is an integer of 1 or more) indicates the mth largest absolute value among the element values ​​of the image vector, the image vector is the result of a linear transformation of a vector whose elements are the optimization variables; The action result vector is a result of applying a predetermined diagonal matrix, whose diagonal components have non-negative values, from the left of the absolute value vector, satisfying the condition that the values ​​indicated by the diagonal components with smaller row numbers are equal to or less than the values ​​indicated by the diagonal components with larger row numbers. Solving device.

2. the solution to the inverse problem obtained by the solution-finding process is a result of DC (Difference of Convex Function) optimization, which is optimization that minimizes the difference between two convex functions; one of the two convex functions is a sum of an L2 convex function that is a product of the maximum value of the elements of the diagonal matrix and the L2 norm of the action result vector, and a function that represents the inverse problem; the other of the two convex functions is a result of subtracting the inner product from the L2 convex function; The solution-finding apparatus according to claim 1 .

3. The DC optimization algorithm is an algorithm that is guaranteed to converge to a stationary point. The solution-finding apparatus according to claim 2 .

4. The algorithm used in the DC optimization, which is guaranteed to converge to a stationary point, is the General Double-Proximal Gradient Algorithm. The solution-finding apparatus according to claim 3 .

5. The linear transformation is expressed as a square matrix: The solution-finding apparatus according to claim 1 .

6. The linear transformation is expressed as a non-square matrix. The solution-finding apparatus according to claim 1 .

7. a control unit that executes a solution-finding process to obtain a solution to an underdetermined inverse problem using regularization in which an inner product of an absolute value vector and an action result vector is used as a regularization term, wherein the absolute value vector is a vector obtained based on an image vector, and satisfies a condition that an element value having an m-th smallest element number (m is an integer of 1 or more) has an absolute value that is the m-th largest among the element values ​​of the image vector, the image vector is a result of linear transformation of a vector having optimization variables as elements, and the action result vector is a result of applying, from the left of the absolute value vector, a predetermined diagonal matrix whose diagonal components are non-negative, and which satisfies a condition that a diagonal component with a smaller row number has an equal to or smaller than a diagonal component with a larger row number, a control step for executing the solution-finding process; A solution method having

8. A program for causing a computer to function as the solution-finding device according to any one of claims 1 to 5.

Images