Information processing method and information processing device

Abstract

This information processing method comprises a control step in which a computer learns an objective function in an image linear inverse problem to be analyzed, wherein the learning uses labeled data in which a label indicates a correct answer of the image linear inverse problem, the objective function in the learning is updated so as to reduce the difference between the label and a result obtained by using the objective function as a solution of the image linear inverse problem for data to which the label is given, the objective function is a function obtained by adding a zero-th function to the result obtained by subtracting a second function from a first function, the zero-th function is a differentiable convex function, the first function is a convex function capable of proximal mapping, and the second function is a convex function which is capable of proximal mapping and of which the result obtained by subtracting from the first function becomes a normalization function.

Description

Information processing method and information processing apparatus

[0001] This invention relates to an information processing method and an information processing apparatus.

[0002] Non-convex programming problems include techniques for reconstructing hyperspectral images, as well as solving linear inverse problems of images such as super-resolution, colorization, and deconvolution.

[0003] Sogabe, Yoko. "Compressed spectral imaging using wavelength-dependent PSF metalens." The Journal of the Institute of Image Information and Television Engineers 76.2 (2022): 234-239.

[0004] One technique for solving linear inverse problems involving images is the General double-proximal gradient algorithm for DC programming. In the General double-proximal gradient algorithm for DC programming, the objective function is the difference between a first convex function that can be mapped by proximity and a second convex function that can be mapped by proximity, plus a differentiable function. Therefore, the General double-proximal gradient algorithm for DC programming is a technique for solving linear inverse problems involving images using non-convex functions.

[0005] However, when applying the General double-proximal gradient algorithm for DC programming, unless the first and second convex functions are appropriately set in advance, it is not possible to obtain a suitable solution to the linear inverse image problem to be solved. In other words, unless the objective function is appropriately set in advance, it is not possible to obtain a suitable solution to the linear inverse image problem to be solved. A suitable solution means a solution in which the difference from the true value is within a predetermined range. That is, a suitable solution means a solution with the desired accuracy.

[0006] The appropriate settings, that is, the settings that yield a solution with the desired accuracy, are obtained through trial and error for each image linear inverse problem to be solved. Therefore, the burden of solving image linear inverse problems was sometimes considerable.

[0007] In view of the above circumstances, the present invention aims to provide a technology that reduces the burden required to solve image linear inverse problems.

[0008] One aspect of the present invention is an information processing method comprising: a control step in which a computer learns an objective function in an image linear inverse problem to be analyzed, wherein the learning is performed using labeled data in which the labels indicate the correct solution to the image linear inverse problem, and in the learning, the objective function is updated to minimize the difference between the result obtained using the objective function as the solution to the image linear inverse problem for the data to which the labels are assigned and the labels, wherein the objective function is a function obtained by adding the result obtained by subtracting the second function from the first function, the zero function is a differentiable convex function, the first function is a convex function that can be mapped by proximity, and the second function is a convex function whose result obtained by subtracting it from the first function is a regularization function.

[0009] One aspect of the present invention is an information processing device comprising a control unit for learning an objective function in an image linear inverse problem to be analyzed, wherein the learning is performed using labeled data in which the labels indicate the correct solution to the image linear inverse problem, and in the learning, the objective function is updated to minimize the difference between the result obtained using the objective function as the solution to the image linear inverse problem for the data to which the labels are assigned and the labels, wherein the objective function is a function obtained by adding the result obtained by subtracting the second function from the first function, the zero function is a differentiable convex function, the first function is a convex function that can be mapped by proximity, and the second function is a convex function whose result when subtracted from the first function is a regularization function.

[0010] This invention makes it possible to reduce the burden required to solve image linear inverse problems.

[0011] An explanatory diagram illustrating the information processing device of the embodiment. A diagram showing an example of the hardware configuration of the information processing device 1 in the embodiment. A flowchart showing an example of the processing flow executed by the information processing device 1 in the embodiment.

[0012] (Embodiment) Figure 1 is an explanatory diagram illustrating an information processing device 1 of an embodiment. The information processing device 1 includes a control unit 11 which has a processor 91 such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), or NPU (Neural Network Processing Unit) connected by a bus, and a memory 92, and executes a program.

[0013] The control unit 11 performs a learning process. The learning process involves learning the objective function of the linear inverse problem of the image being analyzed. That is, learning is performed with the objective function of the linear inverse problem of the image being analyzed as the learning target. The linear inverse problem of the image being analyzed may be, for example, hyperspectral image reconstruction, super-resolution, colorization, or deconvolution.

[0014] To aid in understanding the information processing device 1, let me explain what it means to solve an inverse problem. Solving an inverse problem, including the linear image inverse problem, is a technique for estimating the cause that produced a result based on the result itself. Here, the "cause" is the solution in the inverse problem. For the sake of simplicity, the above result in the inverse problem will be referred to as the "foundation information." I want to emphasize that the foundation information is not the result of solving the inverse problem. The foundation information is the input in the inverse problem, not the output. The above cause, not the result, is the result of solving the inverse problem, i.e., the output in the inverse problem. To explain the relationship with the forward problem, in the forward problem, the foundation information is obtained based on the true value of the above cause.

[0015] Now, let's explain how inverse problems are solved in detail. When solving the inverse problem of the object being analyzed, we minimize or maximize the objective function. The value that minimizes or maximizes the objective function is the solution to the inverse problem of the object being analyzed. This value can be a scalar, a vector, or a tensor.

[0016] Whether the objective function is minimized or maximized depends on how the objective function is defined, and it is predetermined whether minimizing or maximizing the objective function yields the solution to the inverse problem. More precisely, it is predetermined whether the value obtained by minimizing the objective function is the value obtained as the solution to the inverse problem, or whether the value obtained by maximizing the objective function is the value obtained as the solution to the inverse problem.

[0017] Solving the inverse problem of the object being analyzed is as follows: the learning process is the process of learning the objective function when solving the linear inverse problem of images. More specifically, the learning process is the process of learning the objective function that is the target of minimization or maximization when solving the linear inverse problem of images. Through learning, the objective function is updated so that the value that minimizes or maximizes the objective function is closer to the true value of the linear inverse problem of the object being analyzed.

[0018] The learning process involves using labeled data where the labels represent the correct solutions to the linear inverse image problem being analyzed. In this learning process, the objective function is updated to minimize the difference between the result obtained using the objective function as the solution to the linear inverse image problem for the data to which the labels are assigned and the labels themselves. In other words, the objective function is updated to minimize the difference between the value that minimizes or maximizes the objective function and the labels themselves. Here, the value that minimizes or maximizes the objective function is an estimate of the true value of the solution to the linear inverse image problem being analyzed when the underlying information is input. The data to which the labels are assigned represents the results in the inverse problem, i.e., the underlying information.

[0019] Incidentally, when there is an objective function defined such that the value that minimizes the objective function is obtained as the solution to the inverse problem to be analyzed, at least one example of an objective function defined such that the value that maximizes it is obtained as the solution to the inverse problem to be analyzed can be easily obtained. This is because it is only necessary to consider the inverse function of the objective function defined such that the value that minimizes the objective function is obtained as the solution to the inverse problem to be analyzed. Therefore, hereinafter, for the sake of simplicity of explanation, the information processing apparatus 1 will be described by taking as an example an objective function defined such that the value that minimizes it is obtained as the solution to the inverse problem to be analyzed. Also, hereinafter, the value that minimizes the objective function will be referred to as the minimum value.

[0020] <Regarding the objective function> Now, let's explain the objective function in more detail. The objective function of the learning target in the learning process is, for example, a function obtained by adding the zeroth function to the result of subtracting the second function from the first function. The zeroth function is a differentiable convex function.

[0021] The first function is a proximable convex function. The second function is a proximable convex function, and the convex function obtained by subtracting it from the first function serves as a regularization function. In learning, at least the first function and the second function are parameterized, and the values of these parameters are updated in learning.

[0022] Incidentally, the objective function that satisfies the condition that the zeroth function in the learning process is a function obtained by adding it to the result of subtracting the second function from the first function can be any function as long as this condition is met, and the physical meanings of the zeroth function, the first function, and the second function depend on the image linear inverse problem to be analyzed. However, for the sake of facilitating the understanding of the information processing apparatus 1, it may be advisable to explain an example of the physical meaning, so here an example of the physical meaning of the result of subtracting the second function from the first function will be explained.

[0023] The result of subtracting the second function from the first function represents, for example, a predetermined constraint condition in the image linear inverse problem to be analyzed. This constraint condition represents, for example, a condition that limits candidates for the solution of the image linear inverse problem to be analyzed. Therefore, the constraint condition may be, for example, a condition that a sandstorm image is not adopted as the solution of the image linear inverse problem to be analyzed. This is an example of a physical meaning.

[0024] <Explanation Using Mathematical Formulas> An example of the learning process will be described using mathematical formulas. The objective function, which is a function obtained by adding a function to the result of subtracting the second function from the zeroth function, is expressed by the following formula (1).

[0025]

[0026] To obtain the solution to this image linear inverse problem, it is only necessary to minimize this objective function. Expressing this in a formula gives the following formula (2).

[0027]

[0028] g represents the basis information. g is, for example, a real-valued M-dimensional vector (M is an integer greater than or equal to 1). f is a variable representing the solution to the inverse problem. Therefore, f shows the minimum value when the objective function in formula (1) is minimized. f is, for example, a real-valued N-dimensional vector (N is an integer greater than or equal to 1). φ is a predetermined linear transformation. For example, when g is a real-valued M-dimensional vector and f is a real-valued N-dimensional vector, φ is, for example, an M×N real matrix.

[0029] φ is, for example, an observation matrix when the image linear inverse problem to be analyzed is hyper-spectral image reconstruction. φ is, for example, the product of a blur matrix and a subsampling matrix when the image linear inverse problem to be analyzed is super-resolution. φ is, for example, a color reduction matrix when the image linear inverse problem to be analyzed is colorization. φ is, for example, a convolution matrix when the image linear inverse problem to be analyzed is deconvolution.

[0030] The function R is a proximable convex function. Therefore, the function R is an example of the first function. The function S is a proximable convex function, and the result obtained by subtracting the function S from the function R (i.e., R - S) is a convex function that serves as a regularization function. Therefore, the function S is an example of the second function.

[0031] Also, the first term of equation (1) is an example of the zeroth function.

[0032] The minimization represented by equation (2) is realized by executing the following equations (3) to (5) for k ranging from 0 to Q (Q is a predetermined non - negative integer). In other words, the process of executing equations (3) to (5) from k = 0 to k = Q is equivalent to the process of equation (2).

[0033]

[0034]

[0035]

[0036] h （ｋ） , ２ , １ , （ｋ） is a dual variable. For example, when g is a real - valued M - dimensional vector and f is a real - valued N - dimensional vector, h （ｋ） is, for example, a real - valued N - dimensional vector. u （ｋ） is an auxiliary variable. For example, when g is a real - valued M - dimensional vector and f is a real - valued N - dimensional vector, u （ｋ） is, for example, a real - valued N - dimensional vector.

[0037] H １ （ｋ） and, H ２ （ｋ） and, γ １ （ｋ） and, γ ２ （ｋ） and, θ （ｋ） are all parameters updated by the learning process. That is, H １ （ｋ） and, H ２ （ｋ） and, γ １ （ｋ） and, γ ２ （ｋ） and, θ （ｋ） are all learnable parameters. H １（ｋ） And, H ２ （ｋ） This refers to a mapping that transforms an N-dimensional vector into another N-dimensional vector, for example, when f is a real N-dimensional vector. A mapping is said to have learnable parameters if the mapping contains parameters. γ １ （ｋ） And, γ ２ （ｋ） and θ （ｋ） For example, real numbers.

[0038] As described above, the minimization represented by equation (2) is achieved by executing equations (3) to (5) from k=0 to k=Q (where Q is a predetermined non-negative integer). In the learning process, for example, initial values ​​are given to the learnable parameters, and the data to which the labels will be assigned is input to equations (3) to (5). These equations (3) to (5) are then executed from k=0 to k=Q. Based on the difference between the minimized value obtained as a result of the execution and the labels, the values ​​of the learnable parameters are updated according to predetermined rules regarding the updating of the values ​​of the learnable parameters.

[0039] As described above, the process of executing equations (3) to (5) from k=0 to k=Q is equivalent to the process in equation (2). Therefore, updating the values ​​of the learnable parameters in equations (3) to (5) is equivalent to updating the objective function in equation (1). Equations (3) to (5) are obtained by transforming equation (1) or (2), and the learnable parameters in equations (3) to (5) constitute the parameters of the parameterized R-S.

[0040] The superscript letter (k) indicates that it is used in the calculations of equations (3) to (5) when k = k. Therefore, for example, h （ｋ１） This refers to the dual variable used in the calculations of equations (3) to (5) when k = k1 (where k1 is an integer between 0 and Q, inclusive). （ｋ１） H represents the auxiliary variable used in the calculations of equations (3) to (5) when k = k1. １ （ｋ１） And, H ２ （ｋ１） And, γ １ （ｋ１） And, γ２ （ｋ１） and θ （ｋ１） These all refer to learnable parameters used in the calculations of equations (3) to (5) when k = k1.

[0041] When a predetermined condition for the termination of learning (hereinafter referred to as the "learning termination condition") is met, updates to the learnable parameters cease. The learning termination condition may be, for example, a condition that the values ​​of the learnable parameters are updated a predetermined number of times, or a condition that the change resulting from the update of the values ​​of the learnable parameters is less than a predetermined change.

[0042] By using the learnable parameter values ​​at the point when the learning termination condition is met (hereinafter referred to as "learned parameter values"), the solution to the linear inverse problem of the image being analyzed can be estimated with higher accuracy than before learning was performed.

[0043] Incidentally, the objective function in equation (1) is an objective function that can be minimized by the GDPGDC (general double-proximal gradient algorithm for DC programming) algorithm. Therefore, the process of obtaining the solution to the linear inverse problem of the image being analyzed using the trained parameter values ​​may be, for example, a process of executing the GDPGDC algorithm. Note that the GDPGDC algorithm is an algorithm that guarantees convergence to a stationary point.

[0044] The process of obtaining a solution to the linear inverse problem of the image being analyzed using the learned parameter values ​​(hereinafter referred to as the "solution acquisition process") may be, for example, a process of executing equations (3) to (5), in which the learned parameter values ​​are substituted as the values ​​of the learnable parameters, from k=0 to k=Q.

[0045] The solution acquisition process is performed, for example, by the control unit 11. However, it is not necessarily required that the control unit 11 perform this process. For example, another device different from the information processing device 1 may acquire the learned parameter values ​​obtained by the control unit 11 and use them to obtain the solution to the linear inverse problem of the image being analyzed.

[0046] <Effects of the Learning Process> By executing the learning process, the objective function can be obtained without manual trial and error. The obtained objective function is the result of updating it so that the difference from the label is small. Therefore, the minimized value obtained using this objective function is closer to the true value of the solution to the image linear inverse problem being analyzed than the minimized value obtained using the objective function before the update. Thus, those who want to obtain a solution to the image linear inverse problem can obtain a solution with the desired accuracy simply by setting the learning termination condition according to the desired accuracy. In this way, executing the learning process reduces the burden required to solve the image linear inverse problem.

[0047] <Example of Hardware Configuration of Information Processing Device 1> Figure 2 shows an example of the hardware configuration of the information processing device 1 in the embodiment. The information processing device 1 includes a control unit 11 and executes a program. The information processing device 1 functions as a device comprising a control unit 11, an interface unit 12, and a storage unit 13 by executing a program.

[0048] More specifically, the processor 91 reads the program stored in the storage unit 13 and stores the read program in the memory 92. By executing the program stored in the memory 92, the information processing device 1 functions as a device comprising a control unit 11, an interface unit 12, and a storage unit 13.

[0049] The control unit 11 controls the operation of each functional unit of the information processing device 1. The control unit 11 may, for example, perform a learning process. The control unit 11 may, for example, acquire information stored in the memory unit 13. Specifically, the process of acquiring information stored in the memory unit 13 is a read operation. The control unit 11 may, for example, perform a solution acquisition process.

[0050] The interface unit 12 is configured to include a communication interface for connecting the information processing device 1 to an external device. The interface unit 12 communicates with the external device via wired or wireless means.

[0051] When the control unit 11 executes the learning process, the external device is, for example, the device that transmits the learning data used in the learning process. In such a case, the interface unit 12 obtains the learning data used in the learning process by communicating with the learning data device. Specifically, the learning data is labeled data in which the labels indicate the correct answer to the linear inverse problem of the image being analyzed.

[0052] The external device may be, for example, a device that performs solution acquisition processing. In such a case, the device that performs solution acquisition processing obtains the learned parameter values ​​obtained in the learning process via the interface unit 12. The device that performs solution acquisition processing can then perform solution acquisition processing using the acquired learned parameter values.

[0053] The interface unit 12 may include input devices such as a mouse, keyboard, touch panel, and microphone. The interface unit 12 may also be configured as an interface connecting these input devices to the information processing device 1. In this way, the input devices of the interface unit 12 receive various types of information or signals to the information processing device 1 via wired or wireless connections. Note that the information or signals do not necessarily have to be input to the communication interface of the interface unit 12, but may also be input to the input devices of the interface unit 12.

[0054] The interface unit 12 outputs various types of information, for example. The interface unit 12 includes, for example, a display device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, or an organic EL (Electro-Luminescence) display, as well as a speaker. The interface unit 12 may be configured as an interface for connecting these display devices or speakers to the information processing device 1. Therefore, the interface unit 12 may output information indicated by information or signals input to the input device of the interface unit 12 in the form of an image or sound.

[0055] The storage unit 13 is configured using a computer-readable recording medium such as a magnetic hard disk drive or a semiconductor memory device. The storage unit 13 stores various information related to the information processing device 1. The storage unit 13 stores various information generated by the operation of the control unit 11, for example. The storage unit 13 may reside, for example, on the cloud.

[0056] Figure 3 is a flowchart showing an example of the processing flow performed by the information processing device 1 in the embodiment. The control unit 11 acquires training data (step S101). That is, the control unit 11 acquires labeled data in which the labels indicate the correct answer to the linear inverse image problem to be analyzed. Next, the control unit 11 performs the training process (step S102).

[0057] In this way, the information processing device 1 performs the learning process. Therefore, as described in the <Effects of the Learning Process> above, the burden required to solve the image linear inverse problem can be reduced.

[0058] (Modification) At least one of the first function and the second function may be represented by a neural network. This neural network may be a noise reducer such as U-Net. Therefore, at least one of the first function and the second function may be a noise reducer such as U-Net. For example, H １ （ｋ） And, H ２ （ｋ） This can be a noise reducer. Also, H １ （ｋ） And, H ２ （ｋ） This can be U-Net.

[0059] The information processing device 1 may be implemented using multiple information processing devices connected to each other via a network. In this case, each process executed by the control unit 11 may be performed in a distributed manner by the multiple information processing devices.

[0060] Furthermore, all or part of the functions of the information processing device 1 may be implemented using hardware such as ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), or FPGA (Field Programmable Gate Array). The program may be recorded on a computer-readable recording medium. Computer-readable recording media include, for example, magnetic disks, magneto-optical disks, optical disks (CD-ROM, DVD-ROM, etc.), portable media such as semiconductor memory (volatile memory, non-volatile memory, etc.) (ROM, RAM, etc.), and storage devices such as hard disks built into computer systems. The program may also be transmitted via a telecommunications line.

[0061] While embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and includes designs and the like that do not depart from the spirit of this invention.

[0062] 1... Information processing device, 11... Control unit, 12... Interface unit, 13... Storage unit, 91... Processor, 92... Memory

Claims

1. An information processing method comprising: a control step in which a computer learns an objective function for an image linear inverse problem to be analyzed, wherein the learning is performed using labeled data in which the labels indicate the correct solution to the image linear inverse problem, the learning is performed in which the objective function is updated to minimize the difference between the result obtained using the objective function as the solution to the image linear inverse problem for the data to which the labels are assigned and the labels, the objective function is a function in which the zero function is added to the result obtained by subtracting the second function from the first function, the zero function is a differentiable convex function, the first function is a convex function that can be mapped by proximity, and the second function is a convex function that can be mapped by proximity and whose result when subtracted from the first function is a regularization function.

2. The information processing method according to claim 1, wherein at least one of the first function and the second function is represented by a neural network.

3. The information processing method according to claim 2, wherein the neural network is a noise reducer.

4. An information processing device comprising: a control unit for learning an objective function in an image linear inverse problem to be analyzed, wherein the learning is performed using labeled data where the labels indicate the correct solution to the image linear inverse problem, the learning is performed by updating the objective function to minimize the difference between the result obtained using the objective function as the solution to the image linear inverse problem for the data to which the labels are assigned and the labels, the objective function being a function in which the zero function is added to the result obtained by subtracting the second function from the first function, the zero function being a differentiable convex function, the first function being a convex function that can be mapped to nearby functions, and the second function being a convex function that can be mapped to nearby functions and whose result when subtracted from the first function is a regularization function.

Images