Super-resolution learning method, super-resolution learning apparatus, and super-resolution learning program

Abstract

To provide a super-resolution learning method, a super-resolution learning apparatus, and a super-resolution learning program capable of suppressing increase in time required for creating a learning model for enhancing the resolution of a three-dimensional physical field.SOLUTION: A super-resolution learning method, a super-resolution learning apparatus, and a super-resolution learning program are configured to process training data including first data that represents a physical field, and second data that represents the physical field at a spatial resolution higher than that of the first data. The method includes: calculating, when the first data is input to a learning model, output data to be output from the learning data; calculating a supervised learning loss based on the output data and the second data; calculating a physical loss based on the output data; adding a result of multiplying the physical loss by a first coefficient to the supervised learning loss to calculate a total loss; executing training of the learning model based on the total loss, to monotonously increase the first coefficient when repeating the training.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a super-resolution learning method, a super-resolution learning apparatus, and a super-resolution learning program.

Background Art

[0002] A technique has been proposed in which a plurality of convolutional neural models learned with a small amount of data are used in combination to generate a plurality of high-resolution image candidates from an original image, and among the generated plurality of high-resolution image candidates, the candidate with the smallest difference from the original image is output as the high-resolution image (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When high-resolutionizing a three-dimensional physical field using the technique disclosed in Patent Document 1, since the size of the data to be learned increases, there is a problem that the time for creating a plurality of learning models tends to increase even with a small amount of data.

[0005] The present invention has been made in view of the above problems. An object thereof is to provide a super-resolution learning method, a super-resolution learning apparatus, and a super-resolution learning program capable of suppressing an increase in the time for creating a learning model for high-resolutionization even when high-resolutionizing a three-dimensional physical field.

Means for Solving the Problems

[0006] In order to solve the above problems, according to a super-resolution learning method, a super-resolution learning device, and a super-resolution learning program according to an aspect of the present invention, teacher data including first data representing a physical field and second data representing the physical field with a higher spatial resolution than the first data is processed. When the first data is input to the learning model, output data output from the learning model is calculated, a teacher learning loss is calculated based on the output data and the second data, and a physical loss is calculated based on the output data. Then, the result of multiplying the physical loss by a first coefficient is added to the teacher learning loss to calculate a total loss, training of the learning model is executed based on the total loss, and when the training is repeated, the first coefficient is monotonically increased.

Effect of the Invention

[0007] According to the present invention, even when high-resolutionizing a three-dimensional physical field, an increase in the time required to create a learning model for high-resolutionizing can be suppressed.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Embodiments for Carrying Out the Invention

[0009] Next, embodiments of the present invention will be described in detail with reference to the drawings. In the description, the same components are denoted by the same reference numerals and redundant descriptions are omitted.

[0010] [Configuration of Super-Resolution Learning Device] FIG. 1 is a block diagram showing the configuration of the super-resolution learning apparatus according to the present embodiment. As shown in FIG. 1, the super-resolution learning apparatus includes an acquisition unit 71, a database 73, a controller 100, and an output unit 400. The controller 100 is connected to the acquisition unit 71, the database 73, and the output unit 400 via a wired or wireless communication path.

[0011] The acquisition unit 71 acquires a series of teacher data used for training the learning model. Here, the teacher data includes first data representing a physical field and second data representing the physical field with a higher spatial resolution than the first data. For example, the first data and the second data are three-dimensional data regarding a space to be calculated in or around a predetermined object. The first data and the second data are a set of data indicating the physical field in voxels included in the space to be calculated by scalar values or vector values. The physical field includes at least any one of a density field, a velocity field, a pressure field, a density field, and a temperature field of a fluid.

[0012] Note that a voxel is the minimum unit of data for expressing a three-dimensional object by a computer. For example, a voxel may be a cube that is a unit element of an orthogonal lattice. Also, a voxel may be another polyhedron. On a computer, the space to be calculated is represented as a set of voxels, and each voxel has a scalar value and a vector value. By expressing the physical field for each voxel by scalar values and vector values, the entire set of voxels can represent the physical field in the entire space to be calculated.

[0013] Since the second data represents the physical field with a higher spatial resolution than the first data, the size of the voxels related to the second data is smaller than the size of the voxels related to the first data.

[0014] In addition, the acquisition unit 71 may acquire information on the arrangement of voxels representing the space to be calculated and information on the arrangement of objects included in the space to be calculated.

[0015] The database 73 stores the information acquired by the acquisition unit 71. The database 73 may store teacher data including the first data and the second data, or may store information on the arrangement of voxels and information on the arrangement of objects.

[0016] Furthermore, the database 73 may store parameters related to the learning model calculated by the controller 100 described later. When the learning model is a neural network, the neural network includes an input layer (or, kernel), an output layer where output values are output, and at least one or more hidden layers provided between the input layer and the output layer, and signals propagate in the order of the input layer, the hidden layer, and the output layer. Each layer of the input layer, the hidden layer, and the output layer is composed of one or more units. The units between the layers are connected, and each unit has an activation function (for example, sigmoid function, rectified linear function, softmax function, etc.). A weighted sum is calculated based on a plurality of inputs to the unit, and the value of the activation function with the sum value as a variable becomes the output of the unit. For example, in machine learning, the weights when calculating the sum in each unit of the neural network are adjusted as parameters related to the learning model.

[0017] The database 73 may store the connection relationship between units in the neural network and the weights when calculating the sum in each unit of the neural network. Hereinafter, the "connection relationship" and "weights" between units are referred to as "parameters".

[0018] Each time the learning model is trained by the controller 100, the database 73 may store the parameters related to the updated learning model received from the controller 100.

[0019] In addition, the database 73 may store parameters related to a discrimination model calculated by a controller 100 described later. The discrimination model may be a neural network. For example, in adversarial machine learning, the weights when calculating the sum in each unit of the neural network are adjusted as parameters related to the discrimination model.

[0020] Each time the controller 100 trains the discrimination model, the database 73 may store the parameters related to the updated discrimination model received from the controller 100.

[0021] The output unit 400 outputs various information generated by the controller 100. For example, the output unit 400 may output the output from the learning model to the outside. In addition, the output unit 400 may output the learning model generated by the controller 100 described later. In particular, the output unit 400 may output the parameters related to the learning model.

[0022] The controller 100 (an example of a control unit or a processing unit) is a general-purpose computer including a CPU (Central Processing Unit), a memory, and an input / output unit. A computer program (super-resolution learning program) for functioning as a part of the super-resolution learning device is installed in the controller 100. By executing the computer program, the controller 100 functions as a plurality of information processing circuits (120, 130, 140, 150) included in the super-resolution learning device. When performing super-resolution learning, it is more preferable that the CPU (Central Processing Unit) is a GPU (Graphics Processing Unit), includes a GPU, or is connected to a GPU.

[0023] The controller 100 includes a loss calculation unit 120, a coefficient setting unit 130, a total loss calculation unit 140, and a training execution unit 150 as a plurality of information processing circuits (120, 130, 140, 150).

[0024] When the loss calculation unit 120 inputs the first data into the learning model, it calculates the output data output from the learning model. Then, the loss calculation unit 120 calculates the supervised learning loss based on the output data and the second data. For example, the loss calculation unit 120 may calculate the mean squared error of the difference between the output data and the second data for each voxel, and calculate the total of the mean squared errors over all the voxels included in the space to be calculated to calculate the supervised learning loss. The smaller the supervised learning loss, the more accurately the learning model reproduces the teacher data.

[0025] In addition, the loss calculation unit 120 calculates a physical loss based on the output data. For example, the loss calculation unit 120 calculates the physical loss based on the physical rules followed by the physical field and the output data. Here, the physical rules may include at least any one of the Navier - Stokes equations, the continuity equation, the Poisson equation, and the boundary conditions. The smaller the physical loss, the more the learning model outputs a physical field that follows the physical rules.

[0026] Furthermore, the loss calculation unit 120 may calculate discrimination data output from the discrimination model when the first data and the output data are input into the discrimination model. Then, the loss calculation unit 120 may calculate an adversarial learning loss based on the discrimination data and the second data.

[0027] The coefficient setting unit 130 sets a first coefficient related to the physical loss and a second coefficient related to the adversarial learning loss based on the number of epochs. Here, the "number of epochs" indicates the number of times the training of the learning model is repeated.

[0028] More specifically, when the training of the learning model, which is executed by the training execution unit 150 described later, is repeated, the coefficient setting unit 130 monotonically increases the first coefficient. Similarly, the coefficient setting unit 130 may monotonically increase the second coefficient when the training of the learning model is repeated.

[0029] FIG. 4 is a diagram showing the change in the coefficient with respect to the number of epochs. As shown in FIG. 4, the first coefficient and the second coefficient monotonically increase as the number of epochs increases.

[0030] In addition, the coefficient setting unit 130 may set the first coefficient to be less than a predetermined threshold when the number of times of training is less than a predetermined number of times. Also, the coefficient setting unit 130 may set the first coefficient to 0 when training is first executed. Further, the coefficient setting unit 130 may set the first coefficient to be 1 or less. The coefficient setting unit 130 may set the first coefficient to 1 when training is finally executed.

[0031] Also, the coefficient setting unit 130 may set the second coefficient to be less than a predetermined threshold when the number of times of training is less than a predetermined number of times. Also, the coefficient setting unit 130 may set the second coefficient to 0 when training is first executed. Further, the coefficient setting unit 130 may set the second coefficient to be 1 or less. The coefficient setting unit 130 may set the second coefficient to 1 when training is finally executed.

[0032] In addition, the coefficient setting unit 130 may set the first coefficient to be greater than or equal to the second coefficient.

[0033] The total loss calculation unit 140 calculates the total loss by adding the result of multiplying the physical loss by the first coefficient to the teacher learning loss. Also, the total loss calculation unit 140 may calculate the total loss by further adding the result of multiplying the adversarial learning loss by the second coefficient to the teacher learning loss. That is, the total loss calculation unit 140 may use the sum of the result of multiplying the physical loss by the first coefficient and the teacher learning loss as the total loss, or may use the sum of the result of multiplying the physical loss by the first coefficient, the result of multiplying the adversarial learning loss by the second coefficient, and the teacher learning loss as the total loss.

[0034] The training execution unit 150 executes the training of the learning model based on the total loss. At this time, the training execution unit 150 adjusts the parameters related to the learning model in the direction of reducing the total loss. In particular, the training execution unit 150 executes the training of the learning model until a predetermined number of epochs is reached. For this purpose, the training execution unit 150 may determine whether the predetermined number of epochs has been reached.

[0035] When performing machine learning to generate a neural network related to the learning model, the first data is input to the input layer of the neural network. At this time, the parameters related to the neural network are adjusted so that the error between the value output from the output layer of the neural network and the second data is reduced.

[0036] For example, in order to minimize the error related to the output of the neural network, a gradient descent method, a stochastic gradient descent method, or the like may be used. Here, for the gradient calculation in the gradient descent method and the stochastic gradient descent method, the error backpropagation method may be used.

[0037] In addition, in machine learning by neural networks, generalization performance (discrimination ability for unknown data) and overfitting (a phenomenon in which the model fits well to the data used to create the learning model while the generalization performance does not improve) may become problems.

[0038] Therefore, in order to alleviate overfitting, techniques such as regularization that restrict the degree of freedom of the weights during learning may be used. In addition, techniques such as dropout that probabilistically select units in the neural network and invalidate other units may be used. Furthermore, in order to improve the generalization performance, techniques such as data regularization, data standardization, and data augmentation that eliminate the bias in the data may be used.

[0039] In addition, the training execution unit 150 may execute the training of the discrimination model by a predetermined adversarial machine learning. The training execution unit 150 may adjust the parameters related to the discrimination model in the direction of reducing the total loss.

[0040] [Processing Example of Super-Resolution Learning Device] FIG. 2 is a flowchart showing the processing of the super-resolution learning device according to the present embodiment. FIG. 3 is a flowchart showing the processing (modified example) of the super-resolution learning device according to the present embodiment.

[0041] In step S101, the acquisition unit 71 acquires a series of teacher data used for training the learning model.

[0042] In step S103, when the loss calculation unit 120 inputs the first data into the learning model, it calculates the output data output from the learning model.

[0043] In step S105, the loss calculation unit 120 calculates the teacher learning loss based on the output data and the second data. Also, as shown in FIG. 3, in step S106, the loss calculation unit 120 may calculate the adversarial learning loss based on the discrimination data and the second data.

[0044] In step S107, the loss calculation unit 120 calculates the physical loss based on the output data.

[0045] In step S109, the coefficient setting unit 130 sets the first coefficient related to the physical loss based on the number of epochs. Also, as shown in FIG. 3, in step S110, the coefficient setting unit 130 may set the second coefficient related to the adversarial learning loss based on the number of epochs.

[0046] In step S111, the total loss calculation unit 140 calculates the total loss.

[0047] In step S113, the training execution unit 150 executes the training of the learning model based on the total loss.

[0048] In step S115, the training execution unit 150 determines whether or not a predetermined number of epochs has been reached. If the predetermined number of epochs has not been reached (NO in step S115), the process returns to step S103.

[0049] If the predetermined number of epochs has been reached (YES in step S115), in step S117, the output unit 400 outputs the generated learning model. Thereafter, the flowcharts of FIGS. 2 and 3 end.

[0050] [Effects of Embodiment] As described in detail above, according to the super-resolution learning method, super-resolution learning apparatus, and super-resolution learning program according to the present embodiment, teacher data including first data representing a physical field and second data representing the physical field with a higher spatial resolution than the first data is processed. When the first data is input to the learning model, output data output from the learning model is calculated, a teacher learning loss is calculated based on the output data and the second data, and a physical loss is calculated based on the output data. Then, the result of multiplying the physical loss by a first coefficient is added to the teacher learning loss to calculate a total loss, the training of the learning model is executed based on the total loss, and when the training is repeated, the first coefficient is monotonically increased.

[0051] Thereby, even when high-resolutionizing a three-dimensional physical field, an increase in the time required to create a learning model for high-resolutionizing can be suppressed. Furthermore, the computational cost when creating the learning model can be reduced. Also, the physical loss can be reduced together with the teacher learning loss, and it can be guaranteed that the high-resolutionized physical field is a natural physical field that follows a predetermined physical law. In particular, by monotonically increasing the first coefficient as the number of training times of the learning model increases, the effect of the physical loss is gradually introduced, and the phenomenon of learning collapse can also be suppressed.

[0052] Further, the super-resolution learning method, super-resolution learning apparatus, and super-resolution learning program according to the present embodiment may set the first coefficient to be less than a predetermined threshold when the number of times of training is less than a predetermined number of times. Thereby, at the initial stage of training the learning model, the ratio of the physical loss contributing to the total loss is restricted, and the phenomenon that learning collapses can be suppressed.

[0053] Furthermore, the super-resolution learning method, super-resolution learning apparatus, and super-resolution learning program according to the present embodiment may set the first coefficient to 0 when training is first executed. Thereby, at the initial stage of training the learning model, the ratio of the physical loss contributing to the total loss is restricted, and the phenomenon that learning collapses can be suppressed.

[0054] Also, the super-resolution learning method, super-resolution learning apparatus, and super-resolution learning program according to the present embodiment may set the first coefficient to be 1 or less. Thereby, an increase in the contribution of the physical loss compared to the contribution of the teacher learning loss is suppressed, and the phenomenon that learning collapses can be suppressed.

[0055] Furthermore, the super-resolution learning method, super-resolution learning apparatus, and super-resolution learning program according to the present embodiment may set the first coefficient to 1 when training is last executed. Thereby, at the final stage of training the learning model, the contribution of the teacher learning loss and the contribution of the physical loss become equal. As a result, it is guaranteed that the generated learning model can generate a natural physical field that conforms to a predetermined physical law.

[0056] Also, the super-resolution learning method, super-resolution learning apparatus, and super-resolution learning program according to the present embodiment may calculate discrimination data output from the discrimination model when the first data and the output data are input to the discrimination model, and calculate an adversarial learning loss based on the discrimination data and the second data. And the result of multiplying the adversarial learning loss by the second coefficient may be further added to the teacher learning loss to calculate the total loss.

[0057] This makes it possible to suppress an increase in the time required to create a learning model for high-resolution conversion even when three-dimensional physical fields are to be converted to high resolution. Furthermore, the computational cost for creating the learning model can be reduced. Also, the adversarial learning loss can be reduced along with the supervised learning loss, ensuring that the high-resolution physical fields reflect an improvement in accuracy by adversarial machine learning.

[0058] Furthermore, the super-resolution learning method, super-resolution learning apparatus, and super-resolution learning program according to the present embodiment may monotonically increase the second coefficient when repeating training. By monotonically increasing the second coefficient as the number of training times of the learning model increases, the effect of the adversarial learning loss is gradually introduced, and the phenomenon of learning collapse can be suppressed.

[0059] Also, the super-resolution learning method, super-resolution learning apparatus, and super-resolution learning program according to the present embodiment may set the second coefficient to less than a predetermined threshold when the number of times training has been executed is less than a predetermined number of times. Thereby, at the initial stage of training of the learning model, the ratio of the adversarial learning loss contributing to the total loss is restricted, and the phenomenon of learning collapse can be suppressed.

[0060] Furthermore, the super-resolution learning method, super-resolution learning apparatus, and super-resolution learning program according to the present embodiment may set the second coefficient to 0 when training is first executed. Thereby, at the initial stage of training of the learning model, the ratio of the adversarial learning loss contributing to the total loss is restricted, and the phenomenon of learning collapse can be suppressed.

[0061] Also, the super-resolution learning method, super-resolution learning apparatus, and super-resolution learning program according to the present embodiment may set the second coefficient to 1 or less. Thereby, an increase in the contribution of the adversarial learning loss compared to the contribution of the supervised learning loss is suppressed, and the phenomenon of learning collapse can be suppressed.

[0062] Furthermore, the super-resolution learning method, super-resolution learning apparatus, and super-resolution learning program according to the present embodiment may set the second coefficient to 1 when finally executing the training. Thereby, at the final stage of training the learning model, the contribution of the teacher learning loss and the contribution of the adversarial learning loss become equal. As a result, it is guaranteed that the generated learning model reflects the accuracy improvement by adversarial machine learning.

[0063] Also, the super-resolution learning method, super-resolution learning apparatus, and super-resolution learning program according to the present embodiment may set the first coefficient to be greater than or equal to the second coefficient. Thereby, it is guaranteed that the contribution of the physical loss becomes larger than the contribution of the adversarial learning loss. As a result, it is guaranteed that the high-resolution physical field reflects the accuracy improvement by adversarial machine learning. Furthermore, the phenomenon of learning collapse can be suppressed.

[0064] Furthermore, the super-resolution learning method, super-resolution learning apparatus, and super-resolution learning program according to the present embodiment may calculate the physical loss based on the physical rules that the physical field follows and the output data. Thereby, it can be guaranteed that the physical field generated by the learning model is a natural physical field that follows a predetermined physical rule.

[0065] Also, in the super-resolution learning method, super-resolution learning apparatus, and super-resolution learning program according to the present embodiment, the physical rules may include at least any one of the Navier-Stokes equation, the continuity equation, the Poisson equation, and the boundary conditions. Thereby, it can be guaranteed that the physical field generated by the learning model satisfies the equations of fluid dynamics, the heat conduction equation, or the like. High-resolution conversion of the physical field described by a predetermined physical law can be performed by the learning model.

[0066] Furthermore, in the super-resolution learning method, super-resolution learning apparatus, and super-resolution learning program according to the present embodiment, the physical field may include at least any one of a density field, velocity field, pressure field, density field, and temperature field of a fluid. Thereby, high-resolution conversion of the physical field described by a predetermined physical law can be performed by the learning model.

[0067] Each function shown in the above-described embodiment can be implemented by one or a plurality of processing circuits. The processing circuit includes a programmed processor, an electric circuit, etc., and further includes a device such as an application-specific integrated circuit (ASIC) and circuit components arranged to execute the described functions.

[0068] As described above, the content of the present invention has been described along with the embodiment, but it is obvious to those skilled in the art that the present invention is not limited to these descriptions, and various modifications and improvements are possible. It should not be understood that the discussion and drawings forming a part of this disclosure limit the present invention. Various alternative embodiments, examples, and operation techniques will be apparent to those skilled in the art from this disclosure.

[0069] The present invention naturally includes various embodiments and the like not described herein. Therefore, the technical scope of the present invention is defined only by the invention-specific matters according to the reasonable claims based on the above description.

Explanation of Reference Numerals

[0070] 71 Acquisition Unit 73 Database 100 Controller 120 Loss Calculation Unit 130 Coefficient Setting Unit 140 Total Loss Calculation Unit 150 Training Execution Unit 400 Output Unit

Claims

1. First data representing a physical field, second data representing the physical field with a higher spatial resolution than the first data, A super-resolution learning method for controlling a controller into which teacher data including the above is input, The controller, When the first data is input to the learning model, calculates output data output from the learning model, Calculates a teacher learning loss based on the output data and the second data, Calculates a physical loss based on the output data, Adds the result of multiplying the physical loss by a first coefficient to the teacher learning loss to calculate a total loss, Executes training of the learning model based on the total loss, When repeating the training, monotonically increases the first coefficient, Super-resolution learning method.

2. The controller sets the first coefficient to less than a predetermined threshold when the number of times the training is executed is less than a predetermined number of times. The super-resolution learning method according to Claim 1.

3. The controller sets the first coefficient to 0 when first executing the training. The super-resolution learning method according to Claim 1.

4. The controller sets the first coefficient to 1 or less. The super-resolution learning method according to Claim 1.

5. The controller sets the first coefficient to 1 when last executing the training. The super-resolution learning method according to Claim 1.

6. The controller, When the first data and the output data are input to a discrimination model, calculates discrimination data output from the discrimination model, Calculates an adversarial learning loss based on the discrimination data and the second data, Calculates the total loss by further adding the result of multiplying the adversarial learning loss by a second coefficient to the teacher learning loss. The super-resolution learning method according to Claim 1.

7. The controller monotonically increases the second coefficient when repeating the training. The super-resolution learning method according to Claim 6.

8. The controller sets the second coefficient to less than a predetermined threshold when the number of times the training is executed is less than a predetermined number of times. The super-resolution learning method according to Claim 6.

9. The controller sets the second coefficient to 0 when first executing the training. The super-resolution learning method according to Claim 6.

10. The controller sets the second coefficient to 1 or less. The super-resolution learning method according to Claim 6.

11. The super-resolution learning method according to claim 6, wherein the controller sets the second coefficient to 1 when the training is last executed.

12. The super-resolution learning method according to claim 6, wherein the controller sets the first coefficient to be greater than or equal to the second coefficient.

13. The super-resolution learning method according to claim 1, wherein the controller calculates the physical loss based on the physical rules followed by the physical field and the output data.

14. The super-resolution learning method according to claim 13, wherein the physical rules include at least any one of the Navier-Stokes equation, the continuity equation, the Poisson equation, and the boundary conditions.

15. The super-resolution learning method according to any one of claims 1 to 14, wherein the physical field includes at least any one of a fluid density field, a velocity field, a pressure field, a density field, and a temperature field.

16. A first data representing a physical field, A second data representing the physical field with a higher spatial resolution than the first data, A super-resolution learning device comprising a controller to which teacher data including the above is input, The controller, When the first data is input to the learning model, calculates output data output from the learning model, Calculates a teacher learning loss based on the output data and the second data, Calculates a physical loss based on the output data, Adds the result of multiplying the physical loss by a first coefficient to the teacher learning loss to calculate a total loss, Executes training of the learning model based on the total loss, When repeating the training, monotonically increases the first coefficient, Super-resolution learning device.

17. A first data representing a physical field, A second data representing the physical field with a higher spatial resolution than the first data, A super-resolution learning program for execution in a controller to which teacher data including the above is input, When the first data is input to the learning model, a step of calculating output data output from the learning model, A step of calculating a teacher learning loss based on the output data and the second data, A step of calculating a physical loss based on the output data, A step of adding the result of multiplying the physical loss by a first coefficient to the teacher learning loss to calculate a total loss, A step of executing training of the learning model based on the total loss, A step of monotonically increasing the first coefficient when repeating the training, A super-resolution learning program including the above.

Images