Super-resolution learning method, super-resolution learning apparatus, and super-resolution learning program

Abstract

To provide a super-resolution learning method, a super-resolution learning apparatus, and a super-resolution learning program capable of suppressing increase in time required for creating a learning model for enhancing the resolution of a three-dimensional physical field.SOLUTION: A super-resolution learning method, a super-resolution learning apparatus, and a super-resolution learning program are configured to process training data including first data that represents a physical field, and second data that represents the physical field at a spatial resolution higher than that of the first data. The method includes: calculating, when the first data is input to a learning model, output data to be output from the learning data; calculating a loss based on the output data and the second data at each point in the physical field; setting a coefficient based on a distance from an object of interest, at each point in the physical field; summing the results of multiplying the losses by the coefficient in total of the physical field to calculate a total loss; and executing training of the learning model based on the total loss. The coefficient to be set for a second distance, which is larger than a first distance, is smaller than the coefficient to be set for the first distance.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a super-resolution learning method, a super-resolution learning device, and a super-resolution learning program.

Background Art

[0002] There has been proposed a technique of using a plurality of convolutional neural models learned with a small amount of data in combination to generate a plurality of high-resolution image candidates from an original image, and outputting, as a high-resolution image, a candidate having the smallest difference from the original image among the generated plurality of high-resolution image candidates (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When high-resolutionizing a three-dimensional physical field using the technique disclosed in Patent Document 1, since the size of data to be learned increases, there is a problem that the time for creating a plurality of learning models tends to increase even with a small amount of data.

[0005] The present invention has been made in view of the above problems. An object thereof is to provide a super-resolution learning method, a super-resolution learning device, and a super-resolution learning program capable of suppressing an increase in the time for creating a learning model for high-resolutionization even when high-resolutionizing a three-dimensional physical field.

Means for Solving the Problems

[0006] To solve the above problems, according to a super-resolution learning method, a super-resolution learning device, and a super-resolution learning program according to an aspect of the present invention, teacher data including first data representing a physical field and second data representing the physical field with a higher spatial resolution than the first data is processed. When the first data is input to a learning model, output data output from the learning model is calculated, a loss is calculated based on the output data and the second data at each point of the physical field, and a coefficient is set based on the distance from an object of interest at each point of the physical field. A total loss is calculated by summing the result of multiplying the loss by the coefficient over the entire physical field, and training of the learning model is executed based on the total loss. The coefficient set when the distance is a first distance is larger than the coefficient set when the distance is a second distance greater than the first distance.

Advantages of the Invention

[0007] According to the present invention, even when high-resolutionizing a three-dimensional physical field, an increase in the time required to create a learning model for high-resolutionization can be suppressed.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Embodiments for Carrying Out the Invention

[0009] Next, embodiments of the present invention will be described in detail with reference to the drawings. In the description, the same components are denoted by the same reference numerals and redundant description is omitted.

[0010] [Configuration of Super-Resolution Learning Device] FIG. 1 is a block diagram showing the configuration of the super-resolution learning device according to the present embodiment. As shown in FIG. 1, the super-resolution learning device includes an acquisition unit 71, a database 73, a controller 100, and an output unit 400. The controller 100 is connected to the acquisition unit 71, the database 73, and the output unit 400 via a wired or wireless communication path.

[0011] The acquisition unit 71 acquires a series of teacher data used for training the learning model. Here, the teacher data includes first data representing a physical field and second data representing the physical field with a higher spatial resolution than the first data. For example, the first data and the second data are three-dimensional data regarding a space to be calculated in or around a predetermined object. The first data and the second data are a set of data indicating the physical field in the voxels included in the space to be calculated by scalar values or vector values. The physical field includes at least one of a density field, a velocity field, a pressure field, a density field, and a temperature field of a fluid.

[0012] Note that a voxel is the smallest unit of data for expressing a three-dimensional object by a computer. For example, a voxel may be a cube that is a unit element of an orthogonal lattice. Also, a voxel may be other polyhedra. On a computer, the space to be calculated is represented as a set of voxels, and each voxel has scalar values and vector values. By expressing the physical field for each voxel by scalar values and vector values, the entire set of voxels can represent the physical field in the entire space to be calculated.

[0013] Since the second data represents the physical field with a higher spatial resolution than the first data, the size of the voxels related to the second data is smaller than the size of the voxels related to the first data.

[0014] In addition, the acquisition unit 71 may acquire information on the arrangement of voxels representing the space to be calculated and information on the arrangement of objects included in the space to be calculated.

[0015] The database 73 stores the information acquired by the acquisition unit 71. The database 73 may store teacher data including the first data and the second data, or may store information on the arrangement of voxels and information on the arrangement of objects.

[0016] Furthermore, the database 73 may store parameters related to the learning model calculated by the controller 100 described later. When the learning model is a neural network, the neural network includes an input layer (or kernel), an output layer where output values are output, and at least one or more hidden layers provided between the input layer and the output layer, and signals propagate in the order of the input layer, the hidden layer, and the output layer. Each layer of the input layer, the hidden layer, and the output layer is composed of one or more units. The units between the layers are connected, and each unit has an activation function (for example, a sigmoid function, a rectified linear function, a softmax function, etc.). A weighted sum is calculated based on a plurality of inputs to the unit, and the value of the activation function with the sum value as a variable becomes the output of the unit. For example, in machine learning, the weights when calculating the sum in each unit of the neural network are adjusted as parameters related to the learning model.

[0017] The database 73 may store the connection relationship between the units in the neural network and the weights when calculating the sum in each unit of the neural network. Hereinafter, the "connection relationship" and "weights" between the units are referred to as "parameters".

[0018] Each time the learning model is trained by the controller 100, the database 73 may store the parameters related to the updated learning model received from the controller 100.

[0019] The output unit 400 outputs various information generated by the controller 100. For example, the output unit 400 may output the output from the learning model to the outside. In addition, the output unit 400 may output the learning model generated by the controller 100 described later. In particular, the output unit 400 may output the parameters related to the learning model.

[0020] The controller 100 (an example of a control unit or a processing unit) is a general-purpose computer including a CPU (Central Processing Unit), a memory, and an input / output unit. A computer program (super-resolution learning program) for functioning as part of the super-resolution learning device is installed in the controller 100. By executing the computer program, the controller 100 functions as a plurality of information processing circuits (120, 130, 140, 150) included in the super-resolution learning device. In performing super-resolution learning, it is more preferable that the CPU (Central Processing Unit) is a GPU (Graphics Processing Unit), includes a GPU, or is connected to a GPU.

[0021] The controller 100 includes a loss calculation unit 120, a coefficient setting unit 130, a total loss calculation unit 140, and a training execution unit 150 as a plurality of information processing circuits (120, 130, 140, 150).

[0022] When the loss calculation unit 120 inputs the first data into the learning model, it calculates the output data output from the learning model. Then, the loss calculation unit 120 calculates a loss based on the output data and the second data. For example, the loss calculation unit 120 may calculate the mean squared error of the difference between the output data and the second data for each voxel, and use the mean squared error as the loss. The smaller the loss, the more accurately the learning model reproduces the teacher data.

[0023] Here, the loss calculation unit 120 calculates the loss for each voxel. That is, the loss calculation unit 120 calculates the loss based on the output data and the second data at each point in the physical field.

[0024] The coefficient setting unit 130 sets a coefficient for each voxel based on the distance from the object of interest. That is, the coefficient setting unit 130 sets a coefficient at each point in the physical field based on the distance from the object of interest.

[0025] Here, compared with the coefficient set when the distance from the object is the first distance, the coefficient set when the distance from the object is a second distance greater than the first distance is smaller. For example, the coefficient setting unit 130 sets the coefficient such that the coefficient set when the distance from the object is small is smaller than the coefficient set when the distance from the object is large.

[0026] FIG. 3 is a diagram showing the change in the coefficient with respect to the distance from the object. As shown in FIG. 3, the coefficient monotonically decreases as the distance from the object increases.

[0027] Note that the coefficient setting unit 130 may set the coefficient to a first predetermined value when the distance from the object is equal to or less than a first predetermined distance. Also, the coefficient setting unit 130 may set the coefficient to a second predetermined value when the distance from the object is equal to or greater than a second predetermined distance. Here, the second predetermined distance is greater than the first predetermined distance. Also, the second predetermined value is smaller than the first predetermined value. Thereby, the coefficient set in the vicinity of the object can be made large, and the coefficient at a location away from the object can be set small.

[0028] Also, the coefficient setting unit 130 may set the coefficient in a region where the fluid does not pass to 0. Thereby, the coefficient is set to 0 for a region where it is not necessary to reproduce the physical field by the learning model, and in the training of the learning model, it is possible to secure computational resources for executing calculations in a region where it is necessary to reproduce the physical field.

[0029] In addition, the coefficient setting unit 130 may set coefficients based on the distance from the surface of the object. Thereby, the same coefficient can be set for voxels with the same distance from the surface of the object. As a result, the accuracy of reproducing the physical field in the vicinity of the object can be surely adjusted.

[0030] The total loss calculation unit 140 calculates a multiplication result obtained by multiplying the loss by the coefficient for each voxel. Then, the total loss calculation unit 140 calculates the total loss by summing up the multiplication results for all voxels. That is, the total loss calculation unit 140 calculates the total loss by summing up the results of multiplying the loss by the coefficient over the entire physical field.

[0031] Since the coefficient is calculated by the loss calculation unit 120 based on the distance from the object of interest, the loss in the vicinity of the object is greatly reflected in the total loss calculated by the coefficient setting unit 130. On the other hand, the loss in the region away from the object is weakly reflected in the total loss.

[0032] The training execution unit 150 executes training of the learning model based on the total loss. At this time, the training execution unit 150 adjusts the parameters related to the learning model in the direction in which the total loss decreases. In particular, the training execution unit 150 executes training of the learning model until a predetermined number of epochs is reached. Here, the "number of epochs" indicates the number of times the training of the learning model is repeated. For this purpose, the training execution unit 150 may determine whether or not the predetermined number of epochs has been reached.

[0033] When performing machine learning to generate a neural network related to a learning model, the first data is input to the input layer of the neural network. At this time, the parameters related to the neural network are adjusted so that the error between the value output from the output layer of the neural network and the second data becomes small.

[0034] For example, in order to minimize the error regarding the output of the neural network, gradient descent method, stochastic gradient descent method, etc. may be used. Here, for the gradient calculation in the gradient descent method and the stochastic gradient descent method, the backpropagation method may be used.

[0035] In addition, in machine learning by neural networks, generalization performance (discrimination ability for unknown data) and overfitting (a phenomenon in which the model fits well to the data used for creating the learning model while the generalization performance does not improve) can be problems.

[0036] Therefore, in order to alleviate overfitting, techniques such as regularization that restricts the degree of freedom of weights during learning may be used. In addition, techniques such as dropout that probabilistically selects units in the neural network and invalidates other units may be used. Furthermore, in order to improve generalization performance, techniques such as data regularization, data normalization, and data augmentation that eliminate biases in the data may be used.

[0037] [Processing Example of Super-Resolution Learning Device] FIG. 2 is a flowchart showing the processing of the super-resolution learning device according to the present embodiment.

[0038] In step S101, the acquisition unit 71 acquires a series of teacher data used for training the learning model.

[0039] In step S103, the loss calculation unit 120 calculates output data output from the learning model when the first data is input to the learning model.

[0040] In step S105, the loss calculation unit 120 calculates a loss for each voxel based on the output data and the second data.

[0041] In step S107, the loss calculation unit 120 sets a coefficient for each voxel.

[0042] In step S111, the total loss calculation unit 140 calculates the total loss.

[0043] In step S113, the training execution unit 150 executes the training of the learning model based on the total loss.

[0044] In step S115, the training execution unit 150 determines whether or not a predetermined number of epochs has been reached. If the predetermined number of epochs has not been reached (NO in step S115), the process returns to step S103.

[0045] When the predetermined number of epochs has been reached (YES in step S115), in step S117, the output unit 400 outputs the generated learning model. Then, the flowchart in FIG. 2 ends.

[0046] [Effect of Embodiment] As described in detail above, according to the super-resolution learning method, super-resolution learning apparatus, and super-resolution learning program according to the present embodiment, teacher data including first data representing a physical field and second data representing the physical field with a higher spatial resolution than the first data is processed. When the first data is input to the learning model, output data output from the learning model is calculated, a loss is calculated based on the output data and the second data at each point of the physical field, and a coefficient is set based on the distance from the object of interest at each point of the physical field. The result of multiplying the loss by the coefficient is totaled over the entire physical field to calculate a total loss, and the learning model is trained based on the total loss. The coefficient set when the distance is a second distance greater than the first distance is smaller than the coefficient set when the distance is the first distance.

[0047] Thereby, even when high-resolutionizing a three-dimensional physical field, an increase in the time for creating a learning model for high-resolutionizing can be suppressed. Furthermore, the calculation cost when creating the learning model can be reduced. In particular, compared with the region away from the object, the contribution to the total loss can be increased more in the vicinity of the object where it is more necessary to reproduce the physical field with high resolution. As a result, while reducing the calculation cost when creating the learning model, high-resolutionization of the physical field in the vicinity of the object can be realized.

[0048] Also, in the super-resolution learning method, super-resolution learning device, and super-resolution learning program according to this embodiment, the coefficient set when the distance is small may be smaller than the coefficient set when the distance is large. Thereby, compared with the region far from the object, the contribution to the total loss can be increased more in the vicinity of the object where it is more necessary to reproduce the physical field with high resolution. As a result, while reducing the computational cost when creating the learning model, it is possible to achieve high resolution of the physical field in the vicinity of the object.

[0049] Furthermore, the super-resolution learning method, super-resolution learning device, and super-resolution learning program according to this embodiment may set the coefficient to a first predetermined value when the distance is equal to or less than a first predetermined distance. Thereby, the contribution to the loss of the region where the distance to the object is equal to or less than the first predetermined distance can be kept constant, and the accuracy of the physical field reproduced in the vicinity of the object can be surely adjusted.

[0050] Also, the super-resolution learning method, super-resolution learning device, and super-resolution learning program according to this embodiment may set the coefficient to a second predetermined value when the distance is equal to or greater than a second predetermined distance. Thereby, the contribution to the loss of the region where the distance to the object is equal to or greater than the second predetermined distance can be kept constant, and the accuracy of the physical field reproduced in the region far from the object can be surely adjusted.

[0051] Furthermore, the super-resolution learning method, super-resolution learning device, and super-resolution learning program according to this embodiment may set the coefficient in the region where the fluid does not pass to 0. Thereby, the coefficient is set to 0 for the region where it is not necessary to reproduce the physical field by the learning model, and in the training of the learning model, it is possible to secure the computational resources for executing the calculation in the region where it is necessary to reproduce the physical field.

[0052] In addition, the super-resolution learning method, super-resolution learning apparatus, and super-resolution learning program according to the present embodiment may set coefficients based on the distance from the surface of an object. Thereby, the same coefficient can be set for voxels having the same distance from the surface of the object. As a result, the accuracy of reproducing the physical field in the vicinity of the object can be surely adjusted.

[0053] Furthermore, in the super-resolution learning method, super-resolution learning apparatus, and super-resolution learning program according to the present embodiment, the physical field may include at least any one of a fluid density field, velocity field, pressure field, density field, and temperature field. Thereby, high-resolution conversion of the physical field described by a predetermined physical law can be performed by the learning model.

[0054] Each function shown in the above-described embodiment can be implemented by one or a plurality of processing circuits. The processing circuit includes a programmed processor, an electric circuit, etc., and further includes a device such as an application-specific integrated circuit (ASIC) and circuit components arranged to execute the described functions.

[0055] As described above, the content of the present invention has been described along with the embodiment. However, it is obvious to those skilled in the art that the present invention is not limited to these descriptions, and various modifications and improvements are possible. It should not be understood that the discussion and drawings forming part of this disclosure limit the present invention. Various alternative embodiments, examples, and operation techniques will be apparent to those skilled in the art from this disclosure.

[0056] The present invention of course includes various embodiments not described herein. Therefore, the technical scope of the present invention is defined only by the invention specific matters according to the reasonable claims based on the above description.

Explanation of Reference Numerals

[0057] 71 Acquisition unit 73 Database 100 Controller 120 Loss calculation unit 130 Coefficient Setting Unit 140 Total Loss Calculation Unit 150 Training Execution Unit 400 Output Unit

Claims

1. First data representing a physical field, Second data representing the physical field with a higher spatial resolution than the first data, A super-resolution learning method for controlling a controller into which teacher data including the above is input, The controller, When the first data is input into the learning model, calculates output data output from the learning model, Calculates a loss based on the output data and the second data at each point of the physical field, Sets a coefficient based on the distance from the object of interest at each point of the physical field, Calculates a total loss by summing up the result of multiplying the coefficient by the loss over the entire physical field, Executes training of the learning model based on the total loss, The coefficient set when the distance is a second distance greater than the first distance is smaller compared to the coefficient set when the distance is the first distance, Super-resolution learning method.

2. The coefficient set when the distance is small is smaller than the coefficient set when the distance is large. The super-resolution learning method according to claim 1.

3. The controller sets the coefficient to a first predetermined value when the distance is equal to or less than a first predetermined distance. The super-resolution learning method according to claim 1.

4. The controller sets the coefficient to a second predetermined value when the distance is equal to or greater than a second predetermined distance. The super-resolution learning method according to claim 1.

5. The controller sets the coefficient to 0 in a region where the fluid does not pass through. The super-resolution learning method according to claim 1.

6. The controller sets the coefficient based on the distance from the surface of the object. The super-resolution learning method according to claim 1.

7. The physical field includes at least any one of a fluid density field, velocity field, pressure field, density field, and temperature field. The super-resolution learning method according to any one of claims 1 to 6.

8. First data representing a physical field, Second data representing the physical field with a higher spatial resolution than the first data, A super-resolution learning device including a controller into which teacher data including the above is input, The controller, When the first data is input into the learning model, calculates output data output from the learning model, Calculates a loss based on the output data and the second data at each point of the physical field, Sets a coefficient based on the distance from the object of interest at each point of the physical field, Calculate the total loss by summing up the results of multiplying the coefficient by the loss over the entire physical field. Execute the training of the learning model based on the total loss. The coefficient set when the distance is a second distance greater than the first distance is smaller than the coefficient set when the distance is the first distance. Super-resolution learning device.

9. First data representing a physical field, Second data representing the physical field with a higher spatial resolution than the first data, A super-resolution learning program for execution in a controller into which teacher data including the above is input, Calculating output data output from the learning model when the first data is input to the learning model; Calculating a loss based on the output data and the second data at each point in the physical field; Setting a coefficient based on the distance from the object of interest at each point in the physical field; Calculating the total loss by summing up the results of multiplying the coefficient by the loss over the entire physical field; Executing the training of the learning model based on the total loss, and including: The coefficient set when the distance is a second distance greater than the first distance is smaller than the coefficient set when the distance is the first distance. Super-resolution learning program.

Images