Learning method, inference model, noise reduction method, noise reduction program, and noise reduction system

Abstract

Even if sufficient training data cannot be obtained, the system will properly remove noise from the data. The learning method is a method for generating an inference model used to remove noise from data, and includes: a learning acquisition step (S01, S02) of acquiring training data containing noise and generating power-ordered training data from the training data; a noise removal step (S03) of generating noise-removed training data from the power-ordered training data using an inference model under training; a noise addition step (S03) of adding noise to the noise-removed training data to generate noise-added training data; and a training step (S04) of training the machine learning of the inference model using combinations of the power-ordered training data and the noise-added training data as data containing noise.

Description

Technical Field

[0001] The present invention relates to a learning method for generating an inference model used for removing noise from data, the generated inference model, and a noise removal method, a noise removal program, and a noise removal system for removing noise from data using the inference model.

Background Art

[0002] Conventionally, a technique for generating an inference model for image noise removal by machine learning has been proposed. One such technique is Noise2Noise (see, for example, Non-Patent Document 1). According to Noise2Noise, an inference model can be generated without using an image without noise.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Noise2Noise requires multiple images with identical signal components but different noise levels for use in machine learning. However, depending on the target object, acquiring such images may require special processing such as alignment. Alternatively, there are many targets from which such images cannot be obtained. Furthermore, it can be difficult to acquire a sufficient number of images for machine learning. With conventional methods, if a sufficient number of images suitable for machine learning cannot be acquired, it is not possible to generate an appropriate inference model.

[0005] One embodiment of the present invention has been made in view of the above, and aims to provide a learning method, inference model, noise reduction method, noise reduction program, and noise reduction system that can appropriately remove noise from data even when sufficient training data cannot be obtained. [Means for solving the problem]

[0006] To achieve the above objective, a learning method according to one embodiment of the present invention is a learning method for generating an inference model used to remove noise from data having multiple values, and includes: a learning acquisition step of acquiring learning data having multiple values ​​and containing noise, and generating power-removed learning data having a value obtained by raising each of the multiple values ​​of the acquired learning data to a power of a predetermined value; a noise removal step of generating noise-removed learning data from the power-removed learning data generated in the learning acquisition step using an inference model in the process of training; a noise addition step of adding noise based on a predetermined noise to the noise-removed learning data generated in the noise removal step to generate noise-added learning data; and a training step of training the machine learning of the inference model using the combination of the power-removed learning data generated in the learning acquisition step and the noise-added learning data generated in the noise addition step as data containing noise, wherein after the inference model is trained in the training step, the noise removal step generates noise-removed learning data using the trained inference model as an inference model in the process of training, and the noise removal step, the noise addition step, and the training step are repeated.

[0007] In the learning method according to one embodiment of the present invention, power-exponentiation training data and noise-added training data are generated from training data for use in training an inference model. The power-exponentiation training data is data that has been prepared to allow for appropriate machine learning training. The noise-added training data is obtained by removing the noise contained in the power-exponentiation training data using an inference model in the process of training, and then adding data based on a predetermined noise. Therefore, the noise contained in the power-exponentiation training data and the noise-added training data are independent of each other, and the combination of power-exponentiation training data and noise-added training data is appropriate for training an inference model. As a result, the learning method according to one embodiment of the present invention can generate an inference model that can appropriately remove noise.

[0008] Furthermore, by changing the inference model used to generate noise-added training data to one that corresponds to the training stage each time noise-added training data is generated, new noise-added training data can be obtained according to the training stage. That is, from the same training data, new combinations of power-exponential training data and noise-added training data can be obtained according to the training stage. Therefore, the learning method according to one embodiment of the present invention does not require a large amount of training data to train the inference model. Thus, according to the learning method according to one embodiment of the present invention, noise can be appropriately removed from the data even when sufficient training data cannot be obtained.

[0009] In the training data acquisition step, a pre-set value may be used such that the value of the power-expanded training data and the magnitude of the variability of that value are in a linear relationship. This configuration allows for a linear relationship between the value of the power-expanded training data and the magnitude of the variability of that value. Such power-expanded training data is data that allows for more appropriate training of machine learning. Therefore, this configuration allows for the generation of an inference model that is even more appropriate.

[0010] In the noise addition step, weights may be applied to each pre-set noise value according to the value of the denoised training data, and weighted noise may be added to generate noisy training data. Furthermore, in the noise addition step, weights may be applied to pre-set values ​​α and β such that 0.0 < α ≤ 1.0 and -1.0 < β < 1.0 by α × the value of the denoised training data + β. With these configurations, the weighted noise added to the denoised training data can be made closer to the noise that actually occurs. Therefore, with these configurations, an inference model can be generated more appropriately.

[0011] In the training step, each time the inference model is trained, the denoising step may be used to generate denoised training data using the trained inference model as an in-progress inference model. With this configuration, a new combination of power-exponential training data and noise-added training data can be obtained for the same training data with each training iteration. As a result, an even more appropriate inference model can be generated.

[0012] In one of the iterations, the training step may involve training the machine learning of the inference model using the power-expanded training data generated in the training acquisition step, instead of the noisy training data generated in the noise-adding step. This configuration allows for a more appropriate range of values ​​for the data output from the generated inference model, and enables more effective noise removal from the data.

[0013] In the noise addition step, a pre-set noise may be used, depending on the iteration. With this configuration, noise-added learning data, to which various noises have been added depending on the iteration, can be used to train the inference model. As a result, noise can be removed from the data more appropriately.

[0014] In the noise addition step, the same type of noise intended for removal may be used as the pre-defined noise. With this configuration, the noise-added training data, which has the intended noise to be removed added, can be used to train the inference model. As a result, if the noise is the intended type to be removed, the noise can be removed from the data more appropriately.

[0015] The inference model generated by the learning method according to one embodiment of the present invention is itself an invention with a novel configuration. That is, the inference model according to one embodiment of the present invention is an inference model that causes a computer to function by taking data having multiple values ​​as input, performing calculations according to the input, and outputting information, and is generated by the learning method described above.

[0016] To achieve the above objective, a noise removal method according to one embodiment of the present invention is a noise removal method that removes noise from data having multiple values ​​using an inference model used to remove noise from data having multiple values, and includes: a removal acquisition step of acquiring data to be removed for noise removal that has multiple values ​​and is subject to noise removal, and generating powered noise removal target data having a value obtained by raising each of the multiple values ​​of the acquired noise removal target data to a power of a predetermined value; and a noise removal step of generating denoised data using an inference model from the powered noise removal target data generated in the removal acquisition step, and generating denoised result data of the denoised data having a value obtained by raising each of the multiple values ​​of the generated denoised data to a power of the reciprocal of a predetermined value.

[0017] In the noise reduction method according to one embodiment of the present invention, power-expanded noise reduction data, which is used for noise reduction by an inference model, is generated from the noise reduction target data. The power-expanded noise reduction data is data that can be appropriately noise-reduced by the inference model. Therefore, according to the noise reduction method according to one embodiment of the present invention, noise can be appropriately removed from the data.

[0018] In the noise removal acquisition step, a pre-set value may be used such that the value of the data to be denoised after exponentiation has a linear relationship with the magnitude of the variability of that value. With this configuration, it is possible to establish a linear relationship between the value of the data to be denoised after exponentiation and the magnitude of the variability of that value. Such data to be denoised after exponentiation is data that can be denoised more appropriately by the inference model. Therefore, with this configuration, noise can be removed from the data even more appropriately.

[0019] In the noise reduction acquisition step, interference images from coherent waves may be acquired as data to be denoised. With this configuration, noise can be appropriately removed from interference images from coherent waves.

[0020] The noise reduction method further includes an OCT step in which an optical interference image is acquired by OCT, and in the noise reduction acquisition step, the optical interference image acquired in the OCT step may be acquired as data to be noise-reduced. With this configuration, noise can be appropriately removed from the optical interference image acquired by OCT.

[0021] The inference model may be generated by the learning method described above. With this configuration, the inference model generated by the learning method is used to remove noise. Therefore, noise can be appropriately removed from the data.

[0022] One embodiment of this invention can be described as a noise reduction method as described above, but it can also be described as an invention of a noise reduction program and a noise reduction system as follows. These are substantially the same invention, differing only in category, and produce similar functions and effects.

[0023] That is, the noise removal program according to an embodiment of the present invention is a noise removal program that causes a computer to operate as a noise removal system that removes noise from data having a plurality of values using an inference model used for removing noise from data having a plurality of values. The noise removal program causes the computer to acquire noise removal target data that has a plurality of values and is a target for noise removal, and for each of the plurality of values included in the acquired noise removal target data, generates post-power noise removal target data having a value obtained by raising to the power of a preset value. The noise removal program includes a removal acquisition unit that generates post-power noise removal target data, and a noise removal unit that generates noise removal result data for the noise removal target data by generating data after noise removal from the post-power noise removal target data generated by the removal acquisition unit using the inference model, and for each of the plurality of values included in the generated data after noise removal, raising to the power of the reciprocal of the preset value.

[0024] A noise removal system according to an embodiment of the present invention is a noise removal system that removes noise from data having a plurality of values using an inference model used for removing noise from data having a plurality of values. The noise removal system includes a removal acquisition unit that acquires noise removal target data that has a plurality of values and is a target for noise removal, and generates post-power noise removal target data having a value obtained by raising to the power of a preset value for each of the plurality of values included in the acquired noise removal target data, and a noise removal unit that generates noise removal result data for the noise removal target data by generating data after noise removal from the post-power noise removal target data generated by the removal acquisition unit using the inference model, and for each of the plurality of values included in the generated data after noise removal, raising to the power of the reciprocal of the preset value.

[0025] The noise removal system may further include an OCT device that acquires an optical interference image by OCT, and the removal acquisition unit may acquire the optical interference image acquired by the OCT device as a noise removal target.

Advantages of the Invention

[0026] According to one embodiment of the present invention, even when sufficient learning data cannot be obtained, noise can be appropriately removed from the data.

Brief Description of Drawings

[0027] [Figure 1] It is a diagram showing the configuration of a learning system and a noise removal system according to an embodiment of the present invention. [Figure 2] It is a diagram showing an outline of a learning method according to an embodiment. [Figure 3] It is a diagram showing an outline of generation of data used for generation of an inference model. [Figure 4] It is a graph showing the relationship between the luminance value in an image and the standard deviation value. [Figure 5] It is a graph showing the relationship between the luminance value in the image after scale conversion and the standard deviation value. [Figure 6] It is a flowchart showing a learning method which is a process executed in a learning system according to an embodiment of the present invention. [Figure 7] It is a flowchart showing a noise removal method which is a process executed in a noise removal system according to an embodiment of the present invention. [Figure 8] It is an example of an image of a noise removal result by an inference model for each number of repetitions of machine learning training. [Figure 9] It is an example of an image of a noise removal result according to the weighting of noise added during generation of noise-added learning data. [Figure 10] It is an example of an image of a noise removal result for each value of α used for weighting of noise added during generation of noise-added learning data. [Figure 11] It is an example of noise removal performed according to an embodiment. [Figure 12] It is an example of noise removal performed according to an embodiment. [Figure 13] It is an example of noise removal performed according to an embodiment. [Figure 14] It is an example of noise removal performed according to an embodiment. [Figure 15] This is an example of noise reduction performed by the embodiment. [Figure 16] This is an example of noise reduction performed by the embodiment. [Figure 17] This is an example of noise reduction performed by the embodiment. [Figure 18] This figure shows the configuration of a learning program and a noise reduction program according to an embodiment of the present invention, along with a recording medium. [Modes for carrying out the invention]

[0028] Hereinafter, embodiments of the learning method, inference model, noise reduction method, noise reduction program, and noise reduction system according to the present invention will be described in detail with reference to the drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant explanations are omitted.

[0029] Figure 1(a) shows a learning system 10 that performs the learning method according to this embodiment. Figure 1(b) shows a noise reduction system 20 that performs the noise reduction method according to this embodiment. The learning system 10 is a system (device) that generates an inference model (learning model) used to remove noise from data having multiple values. That is, the inference model is a denoising model that denoises data. The noise reduction system 20 is a system (device) that removes noise from data having multiple values ​​using the inference model generated by the learning system 10.

[0030] In this embodiment, the data having multiple values ​​to be denoised is, for example, an image. The image to be denoised may be, for example, an optical interference image (data). An optical interference image is, for example, an image obtained by optical coherence tomography (OCT) or optical diffraction tomography (ODT). The object to be denoised may be an interference image caused by coherent waves, such as the image described above. Coherent waves include laser light and ultrasound. The image to be denoised may be a fluorescence image obtained by imaging with a digital CMOS (Complementary Metal Oxide Semiconductor) camera. The image to be denoised may be an image used for emission analysis (for example, an emission image obtained by imaging with an InGaAs camera).

[0031] The images to be denoised are, for example, images of substrates manufactured on a production line, and the denoised images may be used to check for defects in the substrates. The images to be denoised are also images of cells, and the denoised images may be used to analyze the cells. Furthermore, any image containing noise can be used to denoise, other than those mentioned above.

[0032] Furthermore, data containing multiple values ​​to be removed from noise does not necessarily have to be an image; for example, it may be spectral data or time-series data. Spectral data is, for example, intensity data against wavenumber (wavelength, frequency). Time-series data is, for example, intensity data for each time period, and specifically, it may be output data (output signals) from detectors such as photomultiplier tubes or point sensors, or measurement data of biological information such as hemoglobin concentration.

[0033] The noise contained in the data may be uncorrelated or correlated between multiple values ​​that make up the data (for example, between pixels that make up an image). Uncorrelated noise is, for example, noise that arises randomly from the noise distribution. For example, if the data is acquired from a point sensor, the next noise value cannot be predicted from only one point in the time series. Or, if the data is acquired from a line or 2D sensor, the noise value of adjacent pixels cannot be predicted. Such noise may be, for example, thermal noise, shot noise, or pixel defects or misalignment.

[0034] Correlated noise is noise that is not randomly generated but is related to other noises. For example, a portion of the noise that is extracted may be correlated with noise in other areas. For instance, if the data is acquired from a point sensor, the continuity of the noise's time series can predict the next noise value. Alternatively, if the data is acquired from a line or 2D sensor, the noise value of adjacent pixels can be predicted. Such noises include, for example, periodic noise, speckle noise, or unwanted signal components.

[0035] The noise included in the data may be both the uncorrelated noise and the correlated noise described above. Furthermore, the noise included in the data may be other types of noise. In this embodiment, noise whose brightness changes (different distributions) depending on the object is assumed to be the target of noise reduction. In this embodiment, for example, speckle noise reduction is assumed. However, the target of reduction does not necessarily have to be speckle noise.

[0036] The learning system 10 and the noise reduction system 20 are comprised of a conventional computer including hardware such as a processor, memory, and communication modules. The processor is, for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit).

[0037] The computers comprising the learning system 10 and the noise reduction system 20 may be a computer system including multiple computers. Furthermore, the computers may be configured using cloud computing or edge computing. The functions of the learning system 10 and the noise reduction system 20, as described later, are performed by these components operating through programs or the like.

[0038] Furthermore, the learning system 10 and the noise reduction system 20 may include imaging devices such as cameras for acquiring images used in processing. In this case, the learning system 10 and the noise reduction system 20 are configured to include a computer and an imaging device. For example, the learning system 10 and the noise reduction system 20 may include an OCT device as the imaging device, which acquires optical interference images, which are images used in processing, by OCT. In this case, the learning system 10 uses the optical interference images to generate an inference model. In the noise reduction system 20, the optical interference images are the data to be denoised. Alternatively, the learning system 10 and the noise reduction system 20 may be included in imaging devices such as cameras for acquiring images used in processing. In Figure 1, the learning system 10 and the noise reduction system 20 are shown as separate systems (devices), but they may be implemented by the same system (device).

[0039] Next, the functions of the learning system 10 and noise reduction system 20 according to this embodiment will be described. As shown in Figure 1, the learning system 10 is configured to include a learning acquisition unit 11, a noise reduction unit 12, a noise addition unit 13, and a training unit 14.

[0040] The learning system 10 generates an inference model through machine learning training. The inference model is composed of, for example, a neural network. The neural network may be multi-layered. That is, the inference model may be generated by deep learning. The neural network may also be a convolutional neural network (CNN). The form of the inference model may be the same as conventional inference models generated by N2N (Noise2Noise), for example, a 6-layer Conv2D+ReLU.

[0041] An inference model is used to remove noise from data that has multiple values. For example, an inference model takes data to be denoised as input and outputs denoised data. In this case, the inference model has neurons in its input layer to receive the data to be denoised. For example, the information input to the inference model is the pixel value (luminance value) of each pixel in an image, which is the data to be denoised. In this case, the input layer has neurons equal to the number of pixels in the image, and each neuron is input with the pixel value of the corresponding pixel.

[0042] Furthermore, the inference model includes neurons in the output layer to output denoised data. For example, the information output from the inference model is the pixel value of each pixel in the image, which is the denoised data. In this case, the output layer has as many neurons as there are pixels in the image, and the pixel value of the corresponding pixel is output from each neuron.

[0043] Alternatively, the inference model may take data to be denoised as input and output data containing the noise present in that data. In this case, the denoised data can be obtained by removing the output noise from the data to be denoised. Furthermore, the inference model may be any other type of model that removes noise from the data to be denoised.

[0044] An inference model is designed to allow a computer to take data with multiple values ​​as input, perform calculations based on the input, and output information. Inference models are intended to be used as program modules, which are part of artificial intelligence software. For example, an inference model is used in a computer equipped with a processor and memory, where the computer's processor operates according to instructions from the model stored in memory. For instance, the computer's processor operates according to these instructions, inputting information into the model, performing calculations appropriate to the model, and outputting results from the model. Specifically, the computer's processor operates according to these instructions, inputting information into the input layer of a neural network, performing calculations based on parameters such as learning weighting coefficients in the neural network, and outputting results from the output layer of the neural network. Note that an inference model may be composed of something other than a neural network.

[0045] Figure 2 shows an overview of the learning method for generating the inference model according to this embodiment. Figure 3 shows an overview of the generation of the data used to generate the inference model. In this embodiment, data containing noise is used to generate the inference model, and it is not necessary to use data without noise. The learning method in this embodiment, described below, is called Noise2Prior, in contrast to the conventional Noise2Noise method.

[0046] The learning data acquisition unit 11 is a learning data acquisition means that acquires learning data having multiple values ​​and containing noise, and generates powered learning data having values ​​obtained by raising each of the multiple values ​​in the acquired learning data to a power of a predetermined value. The learning data acquisition unit 11 may use a value as the predetermined value such that there is a linear relationship between the value in the powered learning data and the magnitude of the variation of that value. For example, the learning data acquisition unit 11 acquires learning data and generates powered learning data as follows.

[0047] The training data is in the same format as the data to be denoised. For example, if the data to be denoised is an image, the training data is an image of the same size as that image. The training acquisition unit 11 acquires an image that contains noise and is larger in size than the training data as the training source image (source data). An image containing noise is, for example, an image obtained by imaging a pre-prepared object with an imaging device while the captured image is in a state where noise is present. The source image is acquired, for example, by receiving it from the imaging device that captured the source image, or by accepting a source image input operation from the user to the training system 10.

[0048] The training data acquisition unit 11 extracts and acquires images (image patches) from the original image that are smaller than the original image and of a predetermined size, as training data. The training data acquisition unit 11 may, for example, acquire multiple training data at randomly different positions from a single original image. Alternatively, the training data acquisition unit 11 may, for example, acquire training data (images) from multiple original images (original image pool).

[0049] The training data acquisition unit 11 may acquire training data by methods other than cropping from the original image as described above. The number of training data acquired may be one or multiple.

[0050] The learning data acquisition unit 11 pre-stores a value for generating power-expanded learning data. This value is a pre-set value. For each of the multiple values ​​in the acquired learning data, the learning data acquisition unit 11 generates power-expanded learning data that has a value raised to the power of that value. For example, if the learning data is an image, the learning data acquisition unit 11 raises the pixel values ​​of all pixels constituting the image (which is the learning data) to the power of that value, and generates an image with the resulting powered values ​​as the pixel values, which is then used as power-expanded learning data. In other words, the learning data acquisition unit 11 performs a scale transformation on the learning data.

[0051] The scaling transformation described above is intended to appropriately remove noise from the data, taking into account the characteristics of the noise to be removed in relation to the data values. For example, the scaling transformation takes the following into consideration: Noise in an image has multiple contributing factors. To appropriately remove noise from an image, it is necessary to remove (separate) not only noise that is uniformly present throughout the image, but also noise that changes due to the target object.

[0052] Figure 4 shows a graph illustrating the relationship between luminance values ​​and standard deviation (SD) values ​​in an image. The graph in Figure 4 shows the relationship between the average luminance value in a small region of the image (horizontal axis) and the standard deviation value in that small region (vertical axis). The standard deviation value in a small region is thought to correspond to the magnitude of noise.

[0053] The graph in Figure 4(a) shows the optical interference image obtained by OCT (specifically, the image stored in OCTID (Optical Coherence Tomography Image Database), an open-access database). The noise contained in the optical interference image obtained by OCT is considered to be speckle noise. The graph in Figure 4(b) shows the fluorescence image obtained by imaging with a digital CMOS camera. The noise contained in the fluorescence image obtained by imaging with a digital CMOS camera is considered to be shot noise.

[0054] Shot noise, which occurs in dark fluorescence images, is a statistical fluctuation associated with the detection of fluorescent photons and is proportional to the number of photons. As a result, as shown in the graph in Figure 4(b), the relationship between the luminance value and the standard deviation in fluorescence images is generally linear. Speckle noise is noise caused by the interference of coherent waves. As a result, as shown in the graph in Figure 4(a), the relationship between the luminance value and the standard deviation in optical interference images is generally a power of 1 / d. In the example in Figure 4(a), the standard deviation is approximately 0.5889 times the luminance value.

[0055] When the relationship between pixel values ​​and standard deviation is linear, it tends to be easier to remove noise from the data appropriately. Specifically, when training an inference model using data with this relationship, the model learns properly and can remove noise appropriately. For example, if the relationship between pixel values ​​and standard deviation in the training data is not linear, such as a power relationship, the machine learning training of the inference model may not converge properly or may have poor convergence (i.e., it may be difficult to converge). When the relationship between pixel values ​​and standard deviation is linear, the machine learning training of the inference model improves. This is because it is easier to remember the relationship between pixel values ​​and noise during training, meaning that even a simple model can learn effectively.

[0056] The learning acquisition unit 11 uses a pre-set value for generating the power-expanded learning data such that the value of the power-expanded learning data and the magnitude of the variability of that value (for example, the value of the standard deviation of that value as described above) are in a linear relationship. For example, the power value (1 / d) when the pixel value of the image, which is the learning data, and the variability of the pixel value are in a power relationship may be calculated in advance, and this value may be used as the pre-set value for generating the power-expanded learning data. The value set as described above is such that the value of the power-expanded learning data and the magnitude of the variability of that value are in a linear relationship. In this case, as shown in Figures 2 and 3, the learning acquisition unit 11 performs a 1 / d power scaling transformation on the image 30, which is the learning data, to generate the image 31, which is the power-expanded learning data.

[0057] Figure 5 shows the relationship between the luminance value and the standard deviation (SD value) in the power-expanded training data image when the above scale transformation is applied to the optical interference image related to the graph in Figure 4(a). The value 1 / d used for the scale transformation in this case is 0.5889 (d ≈ 1.7). As shown in Figure 5, in the power-expanded training data image, the relationship between the luminance value and the standard deviation is approximately linear. By using power-expanded training data with such a relationship, noise can be appropriately removed from the data.

[0058] Note that the pre-set values ​​used to generate the power-exponentiated training data do not necessarily have to be the values ​​described above; any value that appropriately removes noise from the data is acceptable. The training acquisition unit 11 outputs the generated power-exponentiated training data to the noise reduction unit 12 and the training unit 14.

[0059] The noise reduction unit 12 is a noise reduction means that generates noise-reduced training data from the power-up training data generated by the training acquisition unit 11, using an inference model in the process of training. After the training unit 14 has trained the inference model, the noise reduction unit 12 generates noise-reduced training data using the trained inference model as an inference model in the process of training. The noise reduction unit 12 may generate noise-reduced training data using the trained inference model as an inference model in the process of training each time the training unit 14 has trained the inference model.

[0060] The noise reduction unit 12 generates training data after noise reduction, for example, as follows. The generation of the inference model is performed by repeatedly processing by the noise reduction unit 12 (noise reduction step), processing by the noise addition unit 13 (noise addition step), and processing by the training unit 14 (training step).

[0061] As described later, the training unit 14 trains the machine learning of the inference model. The training unit 14 outputs the inference model after machine learning training, i.e., the inference model in the middle of training, to the denoising unit 12 during the above iteration. The denoising unit 12 receives the inference model from the training unit 14 and stores it. The output of the inference model from the training unit 14 to the denoising unit 12 may be performed, for example, by sharing the inference model or copying the inference model. The output of the inference model from the training unit 14 to the denoising unit 12 may be performed each time the inference model is trained. Alternatively, the output may be performed not each time the inference model is trained, but at a predetermined timing in the above iteration.

[0062] The denoising unit 12 receives power-reduced training data from the training acquisition unit 11. The denoising unit 12 generates denoised training data (denoised power-reduced training data) from the input power-reduced training data using an inference model in the middle of training. For example, if the inference model outputs a denoised image, as shown in Figures 2 and 3, the denoising unit 12 inputs the scale-transformed image 31, which is the power-reduced training data, into the inference model and obtains the output of the intermediate inference image 32, which is the denoised training data, from the inference model.

[0063] For example, if the inference model outputs noise, the noise reduction unit 12 removes the noise obtained using the inference model from the power-expanded training data to obtain denoised training data. Alternatively, the noise reduction unit 12 may generate denoised training data using methods other than those described above, as long as they utilize an inference model.

[0064] While the inference model during training may not have sufficient noise reduction capabilities, it can still be used for noise reduction. Therefore, the denoised training data generated by the denoising unit 12 will have less noise than the power-expanded training data. The denoising unit 12 outputs the generated denoised training data to the noise addition unit 13.

[0065] The noise addition unit 13 is a noise addition means that adds noise based on pre-set noise to the training data after noise removal generated by the noise removal unit 12 to generate noise-added training data. The noise addition unit 13 may also generate noise-added training data by adding weighted noise for each pre-set noise value according to the value of the training data after noise removal. In this case, the noise addition unit 13 may weight the pre-set values ​​α and β, which are weights set so that 0.0 < α ≤ 1.0 and -1.0 < β < 1.0, by the value of α × the value of the training data after noise removal + β. The noise addition unit 13 may use pre-set noise according to the repetition as the pre-set noise. The noise addition unit 13 may also use noise of the same type as the noise assumed to be removed as the pre-set noise.

[0066] The noise addition unit 13 generates noise-added training data, for example, as follows. The noise-added training data generated by the noise addition unit 13 is used for training the machine learning of the inference model by the training unit 14. The noise addition unit 13 receives the denoised training data from the denoising unit 12. The noise addition unit 13 adds (combines) noise based on pre-set noise to the input denoised training data to generate noise-added training data.

[0067] For example, the noise addition unit 13 stores pre-set noise data for creating noise addition learning data. For example, the noise addition unit 13 acquires an image containing only noise as the original noise image (original data). An image containing only noise is, for example, an image obtained by imaging for noise generation. Imaging for noise generation is imaging by an imaging device in a state where noise is generated in the captured image and there are no objects to be captured. Alternatively, an image containing only noise may be mechanically generated by simulation or the like. The acquisition of the original noise image is performed, for example, by receiving it from the imaging device that captured the original image, or by accepting the user's input operation of the original image to the learning system 10.

[0068] If imaging to generate noise data is performed in the same way as imaging to generate data to be denoised, except for the presence or absence of objects being imaged, then the generated noise data will be of the same type as the noise intended to be removed. This is because if similar imaging is performed, the data is expected to contain the same type of noise. The same applies even if the noise data is generated by means other than imaging. Furthermore, even when noise data is generated by simulation, by performing a simulation that simulates the generation of data to be denoised, the noise data can be of the same type as the noise intended to be removed. Thus, the noise data used for noise-added learning may be of the same type as the noise intended to be removed. However, the noise data used for noise-added learning does not necessarily have to be of the same type as the noise intended to be removed.

[0069] The noise addition unit 13 extracts an image (image patch) from the original image that is the same size as the training data, to be used as noise data for generating noise-added training data. If there is a relationship between the original image of the training data and the original image of the noise data, the noise addition unit 13 may extract the noise data image from a position corresponding to the position of the training data image in the original image. For example, if the original image of the training data and the original image of the noise data are obtained by imaging with the same imaging device (imaging device of the same type), extracting the training data image and the noise image from the same position in the original image may result in a high degree of similarity of the noise contained in them. Such noise is not necessarily appropriate to use as noise to be added. Therefore, the noise addition unit 13 may extract the noise data image from a position different from the position of the training data image in the original image (for example, a position shifted around the position of the training data image where the similarity of the noise is considered to be low (a position with the same distribution control relationship)). The noise addition unit 13 may, for example, acquire noise data at random positions from a single source image each time it is acquired. Alternatively, the noise addition unit 13 may acquire noise data from multiple source images (for example, the speckle noise pool (preparation) shown in Figure 2) (noise selection in Figure 2).

[0070] The noise addition unit 13 may preprocess the noise data. For example, the noise addition unit 13 may divide each pixel value of the noise data image by the average of the pixel values ​​of the entire image to obtain a reference noise. This is to make the noise data more appropriate as noise to be added to the training data after denoising. If preprocessing is performed, the preprocessed noise data will be used as the pre-set noise in subsequent processing.

[0071] The noise addition unit 13 generates noise to be added to the denoised training data (hereinafter referred to as "additional noise") from the pre-set noise described above. As part of generating the additional noise, the noise addition unit 13 weights each of the pre-set noise values ​​according to the values ​​present in the denoised training data. The noise weighting is intended to ensure that the additional noise is appropriate to the denoised training data to which the noise is added, thereby enabling the generated inference model to perform proper noise reduction.

[0072] The noise addition unit 13 performs weighting using the values ​​at the same positions in the pre-set noise and the denoised training data. For example, the noise addition unit 13 uses the pixels at the same positions in the denoised training data image to weight the pixel values ​​of pixels in the pre-set noise image. That is, the denoised training data image is used as a weight image (2D weight map).

[0073] The noise addition unit 13 stores in advance the values ​​of α (slope correction coefficient) and β (low-luminance clipping value), which are set in advance as parameters used for weighting. α and β are values ​​that satisfy 0.0 < α ≤ 1.0 and -1.0 < β < 1.0, respectively. The noise addition unit 13 calculates the weight values ​​to be added to the pre-set noise using the following formula. Weight value = α × value of the training data after noise reduction + β

[0074] The noise addition unit 13 uses the product of the calculated weight value and the pre-set noise value as the noise data for addition. For example, as shown in Figures 2 and 3, the noise addition unit 13 generates a speckle noise brightness adjustment image 34, which is an image in which the pre-set noise image 33 (speckle noise brightness adjustment image) has been weighted using the intermediate inference image 32 and the values ​​of parameters α and β. The speckle noise brightness adjustment image 34 is the noise image for addition. The noise after weighting (speckle noise brightness adjustment image 34) is converted to a contrast that matches the pixel values ​​of the training data after noise removal. The values ​​of parameters α and β should be set to appropriate values ​​through prior tuning or the like.

[0075] Note that the weighting described above does not necessarily have to be applied to the pre-set noise. If weighting is not applied, the pre-set noise described above can be used as additional noise.

[0076] The noise addition unit 13 generates noise-added training data by adding additional noise to the training data after noise removal. For example, as shown in Figures 2 and 3, the noise addition unit 13 adds the pixel values ​​of the intermediate inference image 32, which is the training data after noise removal, and the pixel values ​​of the speckle noise brightness adjustment image 34, which is the additional noise, for each pixel at the same position, to generate a training input image (synthetic input image) 35, which is noise-added training data.

[0077] The noise addition unit 13 may generate noise-added training data by adding noise using noise appropriate to the repetition. For example, the noise addition unit 13 may generate noise-added training data by obtaining different noise data from a speckle noise pool each time noise-added training data is generated.

[0078] Furthermore, the noise addition unit 13 may generate noise-added training data that can be used for machine learning training of the inference model described later, by adding noise based on pre-set noise to the training data after noise removal generated by the noise removal unit 12.

[0079] The training unit 14 is a training means that trains the machine learning of an inference model using a combination of the power-expanded training data generated by the learning acquisition unit 11 and the noise-added training data generated by the noise-adding unit 13, each of which is noise-containing data. In any of the above iterations, the training unit 14 may train the machine learning of the inference model using the power-expanded training data generated by the learning acquisition unit 11 instead of the noise-added training data generated by the noise-adding unit 13.

[0080] The training unit 14 performs machine learning training of an inference model, for example, as follows: The training unit 14 receives power-exponentiation training data from the training data acquisition unit 11. The training unit 14 receives noise-added training data based on the power-exponentiation training data from the noise-adding unit 13. The training unit 14 trains the inference model using a combination of this power-exponentiation training data and noise-added training data.

[0081] The power-up training data input from the training acquisition unit 11 and the noise-added training data input from the noise-adding unit 13 are identical data (for example, the same image) with different noises added to each other. The training unit 14 uses these combinations as training data to train the inference model using machine learning. For example, the training unit 14 trains the inference model using N2N as shown in Non-Patent Literature 1.

[0082] In this case, for example, as shown in Figure 2, the training unit 14 inputs a synthesized input image, which is the data for noise-added training, into the inference model and obtains an inference image, which is the data from which the noise has been removed by the inference model at that point in time. The training unit 14 compares the inference image, which is the data from which the noise has been removed, with the scaled image, which is the data for power-shifted training, and updates the parameters of the inference model by backpropagation based on the loss based on the comparison. In the above example, training is performed by using the data for noise-added training as input to the inference model and comparing it with the output of the power-shifted training model. However, the opposite may also be performed, where the data for power-shifted training is used as input to the inference model and the data for noise-added training is compared with the output of the inference model.

[0083] The training unit 14 trains the inference model, i.e., updates the parameters of the inference model, and then outputs the trained inference model to the denoising unit 12. The denoising unit 12 receives the inference model from the training unit 14 and uses the inference model to generate denoised training data as described above. The training data used to generate the denoised training data (power-shifted training data) may be data that has already been used to train the inference model, or it may be data that has not yet been used to train the inference model. The denoised training data generated by the denoising unit 12 is used in the same way as described above. The training unit 14 uses the combination of training data and noisy training data to train the machine learning model. This process is repeated, and the training of the inference model progresses.

[0084] As mentioned above, the data input to the inference model during training is power-expanded data. Therefore, the denoised data obtained using the output from the inference model is also power-expanded. In order to obtain denoised (non-power-expanded) training data, the denoised data obtained using the output from the inference model needs to be raised to the power of the reciprocal of the power (d) by the training acquisition unit 11. For example, as shown in Figure 2, in order to obtain the final inference image, which is denoised (non-power-expanded) training data, it is necessary to scale the inference image, which is the output from the inference model, by the power of d.

[0085] In any of the above iterations, the training unit 14 may train the inference model using power-law training data instead of noise-added training data. For example, the training unit 14 selects either power-law training data or noise-added training data as the data to be used for training, and trains the inference model using the selected data. If noise-added training data is selected, the training unit 14 trains the inference model as described above.

[0086] If power-expanded training data is selected, the training unit 14 uses only one power-expanded training data to train the inference model's machine learning. In this case as well, the training unit 14 can train the inference model in the same way as when using noisy training data (for example, N2N in the above example).

[0087] For example, the training unit 14 inputs the power-exponentiation training data into the inference model and obtains data with noise removed by the inference model at that point in time. The training unit 14 compares the noise-removed data with the power-exponentiation training data and updates the parameters of the inference model by backpropagation based on the loss calculated from the comparison.

[0088] The noise-added training data described above is obtained by removing noise from the power-law training data using an inference model during training. Because the noise removal function of the inference model during training is not sufficient, the noise-added training data may have had a significant amount of non-noise components removed as well. Therefore, if an inference model is trained using this noise-added training data, the inference model may not be able to output appropriate data. For example, the range of data values ​​output by the inference model may deviate significantly from the range of the noise-removed data.

[0089] Although the post-exponentiation training data contains noise, the range of data values ​​does not deviate significantly from that of properly denoised data. Therefore, by training the inference model using only the post-exponentiation training data as described above, it is possible to prevent the range of data values ​​output by the inference model from deviating significantly from the normal range.

[0090] The choice between using noisy training data or using power-law training data instead of noisy training data may be made probabilistically, for example. Alternatively, power-law training data may be used instead of noisy training data for a predetermined number of iterations, and noisy training data may be used otherwise. Training an inference model using power-law training data instead of noisy training data is, as described above, to bring the values ​​of the data output from the inference model within a certain range. Therefore, the number of such training iterations may be less than the number of training iterations of an inference model using noisy training data for noise reduction, provided that this objective is achieved. For example, the number of such training iterations may be about 10% of the total number of iterations.

[0091] The training of the inference model described above is repeated, similar to conventional machine learning training, for example, until a predetermined number of repetitions or until the generation of the inference model converges based on predetermined conditions. Once the training of the inference model is completed, the training unit 14 outputs the generated inference model to the noise reduction system 20. This completes the functions of the learning system 10.

[0092] As shown in Figure 1, the noise reduction system 20 is configured to include a noise reduction acquisition unit 21 and a noise reduction unit 22.

[0093] The noise removal acquisition unit 21 is a noise removal acquisition means that acquires noise removal target data which has multiple values ​​and is subject to noise removal, and generates powered noise removal target data which has a value obtained by raising each of the multiple values ​​of the acquired noise removal target data to a power of a predetermined value. The noise removal acquisition unit 21 may use a value as the predetermined value such that there is a linear relationship between the value of the powered noise removal target data and the magnitude of the variation of said value. The noise removal acquisition unit 21 may acquire interference images caused by coherent waves as noise removal target data. The noise removal acquisition unit 21 may acquire optical interference images acquired by an OCT device as noise removal target data. For example, the noise removal acquisition unit 21 acquires noise removal target data as follows.

[0094] The noise reduction acquisition unit 21 acquires noise reduction target data by receiving it from the imaging device that captured the image which is the noise reduction target data, or by accepting the user's image input operation to the noise reduction system 20. If the size of the noise reduction target data input to the inference model is smaller than the original image obtained by imaging by the imaging device, similar to the training data described above, the noise reduction acquisition unit 21 may acquire the original image, divide the original image into sizes corresponding to the noise reduction target data input to the inference model, and use the divided images as the noise reduction target data.

[0095] Furthermore, the noise reduction acquisition unit 21 may acquire an optical interference image acquired by an OCT device as noise reduction target data. The noise reduction acquisition unit 21 may also acquire an interference image caused by a coherent wave as noise reduction target data. The above optical interference image and interference image may be generated by conventional methods. The noise reduction acquisition unit 21 may also acquire noise reduction target data by methods other than those described above.

[0096] The noise removal acquisition unit 21 pre-stores a value for generating the power-reduced noise removal data. This value is a pre-set value. For each of the multiple values ​​in the acquired noise removal data, the noise removal acquisition unit 21 generates power-reduced noise removal data having a value raised to that value. For example, if the noise removal data is an image, the noise removal acquisition unit 21 raises the pixel values ​​of all pixels constituting the image (the noise removal data) to that value, and generates an image with the raised values ​​as the pixel values ​​as power-reduced noise removal data. In other words, the noise removal acquisition unit 21 performs a scale transformation on the noise removal data.

[0097] The above scaling transformation is performed for the same purpose as the scaling transformation of training data by the training data acquisition unit 11, and should be performed in the same way as the scaling transformation of training data. The denoising acquisition unit 21 uses a preset value for generating the data to be denoised after exponentiation, such that there is a linear relationship between the value of the data to be denoised and the magnitude of the variability of that value (for example, the standard deviation of that value). For example, if the above relationship is the same for training data and data to be denoised, the preset value for generating the data to be denoised after exponentiation may be the same value as the preset value used by the training data acquisition unit 11 for generating the training data after exponentiation. If the above relationship is different for training data and data to be denoised, the preset value for generating the data to be denoised after exponentiation may be a different value from the preset value used by the training data acquisition unit 11 for generating the training data after exponentiation.

[0098] Note that the pre-set values ​​used to generate the data to be denoised after exponentiation do not necessarily have to be the values ​​described above; any value that appropriately removes noise from the data is acceptable. The denoising acquisition unit 21 outputs the generated data to be denoised after exponentiation to the noise removal unit 22.

[0099] The noise reduction unit 22 is a noise reduction means that generates noise-reduced data from the power-expanded noise-reduced data generated by the noise reduction acquisition unit 21 using an inference model, and generates noise-reduced result data of the noise-reduced data, in which each of the multiple values ​​in the generated noise-reduced data has a value obtained by raising it to the reciprocal of a preset value. The inference model may be one generated by the learning system 10. In this case, the noise reduction unit 22 inputs and stores the inference model generated by the learning system 10 and uses it for noise reduction.

[0100] The noise reduction unit 22 receives the power-expanded noise reduction target data from the noise reduction acquisition unit 21. The noise reduction unit 22 inputs the power-expanded noise reduction target data into the inference model to generate noise reduction data. For example, if the inference model outputs noise reduction data, the noise reduction unit 22 acquires the output from the inference model as noise reduction data. Alternatively, if the inference model outputs noise, the noise reduction unit 22 removes the noise obtained using the inference model from the power-expanded noise reduction target data to obtain noise reduction data. The noise reduction unit 22 may also generate noise reduction data by methods other than those described above, as long as they utilize an inference model.

[0101] The data input to the inference model by the denoising unit 22 is the data to be denoised after exponentiation. Therefore, the denoised data obtained using the output from the inference model will have been exponentiated. In order to obtain the denoised result data, which is the denoised data (without exponentiation) after denoising, it is necessary to raise the denoised data obtained using the output from the inference model by the reciprocal (d) of the exponentiation performed by the denoising unit 22.

[0102] The noise reduction unit 22 stores a value in advance for generating the noise reduction result data. This value (d) is a pre-set value and is the reciprocal of the value (1 / d) used for powering by the removal acquisition unit 21. The noise reduction unit 22 generates noise reduction result data in which each of the multiple values ​​in the noise reduction data obtained using the output from the inference model has a value raised to the power of this value. For example, if the data to be noise reduced is an image, the learning acquisition unit 11 raises the pixel values ​​of all pixels constituting the image, which is the noise reduction data, to the power of this value and generates an image with the raised values ​​as the pixel values ​​as the noise reduction result data. In other words, the noise reduction unit 22 performs an inverse scale transformation on the noise reduction data.

[0103] If the data to be denoised is an optical interference image, the inference model may be one for removing speckle noise from the optical interference image. Since optical interference images may contain speckle noise as the noise to be removed, in this case, the denoising unit 22 uses the inference model to remove the speckle noise from the optical interference image. Furthermore, if the data to be denoised is the above-mentioned optical interference image or interference image, the inference model used by the denoising unit 22 to perform appropriate denoising may be an inference model generated using the same type of image (the above-mentioned optical interference image or interference image) as training data. The inference model used by the denoising unit 22 may be any inference model used to remove noise from data having multiple values, and does not necessarily have to be an inference model generated by the learning system 10.

[0104] The noise reduction unit 22 outputs the generated noise reduction result data. The output of the noise reduction result data can be done in the same way as conventional methods, depending on the purpose of using the data. The above describes the functions of the noise reduction system 20.

[0105] When training a machine learning model using N2N, if the input data for the inference model is the same as the training data used for the output of the inference model, the inference model may not be able to properly remove noise. On the other hand, in N2N machine learning, if the noise contained in the input data and the training data are different from each other, the inference model will be able to properly remove noise.

[0106] In this embodiment, the data used to train the inference model consists of power-law training data and noise-added training data. As described above, the noise-added training data is obtained by denoising the power-law training data and adding other noise. Therefore, the power-law training data and the noise-added training data are designed so that they do not contain common noise and contain different noise from each other. Accordingly, this embodiment enables appropriate noise removal.

[0107] Next, the processes performed by the learning system 10 and noise reduction system 20 according to this embodiment (the operation methods performed by the learning system 10 and noise reduction system 20) will be explained using the flowcharts in Figures 6 and 7.

[0108] First, the learning method, which is a process performed by the learning system 10 according to this embodiment, will be explained using the flowchart in Figure 6. In this process, the learning acquisition unit 11 acquires learning data that has multiple values ​​and contains noise (S01, learning acquisition step). Subsequently, the learning acquisition unit 11 generates powered learning data in which each of the multiple values ​​in the acquired learning data has been raised to a power of a predetermined value (S02, learning acquisition step).

[0109] Next, the denoising unit 12 uses the inference model in the middle of training to generate denoised training data from the power-expanded training data (S03, denoising step). Subsequently, the noise addition unit 13 adds noise based on a predetermined noise to the denoised training data to generate noisy training data (S04, noise addition step). Finally, the training unit 14 uses the combination of the power-expanded training data and the noisy training data as data containing noise to train the machine learning of the inference model (S05, training step).

[0110] Next, the training unit 14 determines whether or not to terminate the training of the inference model (S06). If it is determined not to terminate the training of the inference model (NO in S06), the processes of generating denoised training data (S03), generating noisy training data (S04), and training the machine learning of the inference model (S05) are repeated. In this case, the inference model trained by the training unit 14 is used to generate the denoised training data. After the machine learning training (S05), the decision of whether or not to terminate the training of the inference model (S06) is made again.

[0111] In the determination of whether or not to terminate the training of the inference model (S06), if it is determined that the training of the inference model should be terminated (YES in S06), the inference model generated by the training is output from the learning system 10 to the noise reduction system 20 (S07). The noise reduction system 20 stores the inference model and uses it in the following noise reduction processing. The above is the learning method which is the processing performed by the learning system 10 according to this embodiment.

[0112] Next, the noise reduction method, which is a process performed by the noise reduction system 20 according to this embodiment, will be explained using the flowchart in Figure 7. In this process, the noise reduction acquisition unit 21 acquires noise reduction target data that has multiple values ​​and is subject to noise reduction (S11, noise reduction acquisition step). If the noise reduction system 20 further includes an OCT device, an optical interference image is acquired by the OCT of the OCT device (OCT step), and the noise reduction acquisition unit 21 acquires the optical interference image acquired by the OCT device as noise reduction target data (S11). Subsequently, the noise reduction acquisition unit 21 generates powered noise reduction target data, which has a value raised to a predetermined value for each of the multiple values ​​of the acquired noise reduction target data (S12, noise reduction acquisition step).

[0113] Next, the noise reduction unit 22 generates noise-reduced data from the power-expanded noise-reduced data using an inference model (S13, noise reduction step). Subsequently, the noise reduction unit 22 generates noise-reduced result data for the noise-reduced data, where each of the multiple values ​​in the generated noise-reduced data has a value that is raised to the power of a predetermined value (S14, noise reduction step). The noise-reduced result data is output from the noise reduction unit 22 to a predetermined output destination (S15). The above is the noise reduction method, which is the process performed by the noise reduction system 20 according to this embodiment.

[0114] In this embodiment, power-expanded training data and noise-added training data are generated from training data for training an inference model. The power-expanded training data is data that has been prepared to enable appropriate machine learning training. For example, by using power-expanded training data, the convergence of machine learning training of the inference model can be improved. The noise-added training data is obtained by removing the noise contained in the power-expanded training data using an inference model during training, and then adding data based on a predetermined noise. Therefore, the noise contained in the power-expanded training data and the noise-added training data are independent of each other, and the combination of training data and noise-added training data is appropriate for training the inference model as described above. As a result, in this embodiment, an inference model that can appropriately remove noise can be generated.

[0115] Furthermore, by changing the inference model used to generate noise-added training data during training to one that corresponds to the training stage each time noise-added training data is generated, new noise-added training data can be obtained according to the training stage. In other words, from the same training data, new combinations of power-order training data and noise-added training data can be obtained according to the training stage. Thus, in this embodiment, sequential update learning can be performed in which the data used for training is updated sequentially according to the training of the inference model.

[0116] Therefore, in this embodiment, a large amount of training data is not required to train the inference model. For example, it does not require multiple images with the same signal components but different noise levels, i.e., teacher pairs, as in N2N. Thus, according to this embodiment, even when sufficient training data cannot be obtained, noise can be appropriately removed from the data.

[0117] Since signal and noise components are mixed, in training conventional inference models such as N2N, the only way to increase the number of different mixes (variations) is to increase the original dataset prepared in advance. For example, if periodic noise is mixed with a signal, and a combination with a different phase is required, new data must be prepared. Moreover, whether the required phase is obtained is a matter of chance. In this embodiment, as described above, new combinations of power-law training data and noise-added training data can be obtained according to the training stage, so even with less data prepared in advance for machine learning training than in conventional methods, Semi-Supervised Learning, which is the training of an appropriate inference model, can be achieved.

[0118] In this embodiment, an inference model can be generated even when sufficient training data cannot be obtained; for example, an inference model can be generated from a single image. Therefore, noise reduction becomes possible even for images of objects that cannot be fixed (living organisms), images of objects that deform (2D images using line sensors and point sensors), or images acquired in the past (images that were not consciously prepared), which was previously impossible. Furthermore, according to this embodiment, an appropriate inference model can be generated with fewer training iterations compared to conventional methods.

[0119] Furthermore, as in this embodiment, the pre-set values ​​used to generate the post-power training data may be such that there is a linear relationship between the values ​​in the post-power training data and the magnitude of the variability of those values ​​(for example, the standard deviation of those values ​​as described above). This configuration makes it possible to establish a linear relationship between the values ​​in the post-power training data and the magnitude of the variability of those values. Such post-power training data is data that allows for more appropriate training of machine learning. Therefore, this configuration allows for the generation of an inference model more appropriately. However, the pre-set values ​​used to generate the post-power training data may be other than those mentioned above, as long as they enable the generation of an appropriate inference model.

[0120] Furthermore, as in this embodiment, weighting may be applied to each preset noise value according to the value of the denoised training data, and weighted noise may be added to generate noisy training data. Moreover, as described above, weighting may be performed using the value α × the value of the denoised training data + β. With these configurations, the weighted noise added to the denoised training data can be made closer to the noise that actually occurs. Therefore, with these configurations, an inference model can be generated more appropriately. In addition, the strength of the noise removal effect can be adjusted by adjusting the values ​​of α and β. However, weighting does not necessarily have to be done using the value α × the value of the denoised training data + β, but can be done according to the value of the denoised training data. Also, weighting itself is not necessarily required.

[0121] Furthermore, as in this embodiment, in the denoising step, the denoising model may be used as the inference model in progress each time the inference model is trained in the training step to generate denoising training data. With this configuration, a new combination of power-exponentiated training data and noise-added training data can be obtained for the same training data with each iteration of training. As a result, an inference model can be generated more appropriately. However, the inference model used to generate the denoising training data does not necessarily have to be the one used each time the inference model is trained in the training step; it may be the one used after the inference model has been trained in the training step.

[0122] Furthermore, as in this embodiment, in one of the iterations, the machine learning training of the inference model may be performed using the power-expanded training data generated in the training acquisition step instead of the noise-expanded training data generated in the noise-expanding step during the training step. This configuration makes it possible to make the range of values ​​of the data output from the generated inference model more appropriate and to remove noise more effectively from the data. However, if the range of values ​​of the data output from the inference model becomes appropriate even when the inference model is trained using only the combination of power-expanded training data and noise-expanded training data, it is not necessarily required to use power-expanded training data instead of noise-expanded training data in the training step.

[0123] Furthermore, as in this embodiment, in the noise addition step, a preset noise may be used, which is set according to the iteration. For example, as described above, a different noise may be obtained from the speckle noise pool for each iteration and used for noise addition. With this configuration, noise-added learning data with various noises added according to the iteration can be used to train the inference model. As a result, noise can be removed from the data more appropriately.

[0124] Furthermore, as in this embodiment, in the noise addition step, noise of the same type as the noise expected to be removed may be used as the pre-set noise. With this configuration, the noise-added training data to which the noise expected to be removed has been added can be used to train the inference model. As a result, if the noise is the type expected to be removed, the noise can be removed from the data more appropriately.

[0125] In this embodiment, power-reduced noise-reduced data, which is used for noise reduction by an inference model, is generated from the data to be denoised. The power-reduced noise-reduced data is data that can be appropriately denoised by the inference model. Therefore, according to this embodiment, noise can be appropriately removed from the data.

[0126] Furthermore, as in this embodiment, the preset values ​​used to generate the data to be denoised after exponentiation may be such that there is a linear relationship between the value of the data to be denoised after exponentiation and the magnitude of the variation of that value (for example, the value of the standard deviation of that value as described above). With this configuration, it is possible to make the value of the data to be denoised after exponentiation and the magnitude of the variation of that value linear. Such data to be denoised after exponentiation is data that allows for more appropriate denoising by the inference model. Therefore, with this configuration, noise can be removed from the data even more appropriately. However, the preset values ​​used to generate the data to be denoised after exponentiation may be other than those described above, as long as they are capable of appropriately removing noise from the data.

[0127] Furthermore, as in this embodiment, interference images from coherent waves may be acquired as data to be denoised. With this configuration, noise can be appropriately removed from interference images from coherent waves. However, the data to be denoised may be other than those described above.

[0128] Furthermore, as in this embodiment, the OCT device may acquire an optical interference image using OCT, and the acquired optical interference image may be acquired as data to be denoised. With this configuration, noise can be appropriately removed from the optical interference image acquired by OCT. However, the data to be denoised may be other than those described above. Also, the OCT device does not have to be included in the noise reduction system 20 according to this embodiment.

[0129] Furthermore, as in this embodiment, the inference model used for noise reduction may be generated by the learning method of this embodiment. With this configuration, the inference model generated by the learning method described above is used to perform noise reduction. As described above, the inference model generated in this embodiment can appropriately remove noise, and with this configuration, noise can be appropriately removed from the data.

[0130] Next, an example of this embodiment will be shown. Figure 8 shows examples of images of the noise reduction results by the inference model for each number of training iterations (training iterations) when generating the inference model. In Figure 8, images of the noise reduction results by the inference model for every 100 iterations are shown. Figure 8 shows examples of images for this embodiment, Comparative Example 1, and Comparative Example 2. In this embodiment, power-exponentiated training data (data after scaling by 1 / d) is used to generate the inference model. In addition, in this embodiment, the noise added when generating the noise-added training data is weighted according to the values ​​of the training data after noise reduction.

[0131] In Comparative Examples 1 and 2, training data is used to generate the inference model without raising the pixel values ​​to a power. Furthermore, in Comparative Examples 1 and 2, the noise added during the generation of noise-added training data is not weighted according to the values ​​of the denoised training data. In Comparative Examples 1 and 2, the noise added during the generation of noise-added training data is obtained by multiplying a pre-set noise pixel value (noise contrast) by a fixed value (fixed adjustment value, fixed magnification). Comparative Example 1 uses a relatively small fixed value, while Comparative Example 2 uses a relatively large fixed value.

[0132] As shown in Figure 8, in this embodiment, compared to Comparative Examples 1 and 2, an inference model that appropriately removes noise from the data is generated at an earlier stage of learning. In other words, stable learning is achieved in this embodiment. In Comparative Example 1, where the adjustment value of the added noise is small, the learning of the inference model has not converged. In this way, learning may not converge when the adjustment value of the added noise is small. In Comparative Example 2, where the adjustment value of the added noise is large, the learning of the inference model appears to be converging, but the convergence is slower than in this embodiment.

[0133] Figure 9 shows an example of an image with noise reduction results according to the weighting of the noise added when generating the noise-added training data. Figure 9 shows an example of the image that is the data to be noise-reduced (original image) and an example of an image with noise reduction results according to this embodiment. In this embodiment, the noise added when generating the noise-added training data is weighted according to the values ​​of the training data after noise reduction (weighting per pixel), and the noise is not weighted according to the values ​​of the training data after noise reduction (fixed weighting 1 and fixed weighting 2).

[0134] In fixed-value weighting 1 and fixed-value weighting 2, the noise added during the generation of noise-added training data is obtained by multiplying a pre-set noise pixel value (noise contrast) by a fixed value (fixed adjustment value, fixed magnification). In fixed-value weighting 1, a fixed value of 2.5 is used (high brightness standard), and in fixed-value weighting 2, a fixed value of 2.0 is used (low brightness standard).

[0135] In the example of pixel-by-pixel weighting, appropriate noise reduction is achieved in both low-luminance and high-luminance areas. With fixed weighting 1, the signal (the part where the imaged object is captured) is blurred in low-luminance areas, but appropriate noise reduction is achieved in high-luminance areas (the image quality is the same as in the example of pixel-by-pixel weighting). With fixed weighting 2, the effect of noise reduction is low in high-luminance areas, but appropriate noise reduction is achieved in low-luminance areas (the image quality is the same as in the example of pixel-by-pixel weighting).

[0136] Figure 10 shows examples of images showing the noise reduction results for each value of α used to weight the noise added when generating noise-added learning data according to this embodiment. Figure 10 is an example of an image obtained by removing speckle noise from a portion of an image stored in OCTID according to this embodiment. Figure 10(a) is the image when α=0.8, Figure 10(b) is the image when α=0.9, and Figure 10(c) is the image when α=1.0. In this way, the strength of the speckle noise reduction effect can be adjusted by adjusting the adjustment parameter α.

[0137] Figures 11 to 17 show images of the noise reduction results according to this embodiment. Figures 11 to 17(a) are the images (original images) that were the data to be denoised. Figures 11 to 17(b) are examples of images of the noise reduction results according to this embodiment. In Figures 11 to 17, the images that were the data to be denoised are all images of age-related macular degeneration, extracted from parts of images stored in OCTID. Also, the images that were the data to be denoised in Figures 11 to 17 are images that were not used in the generation (training) of the inference model. Noise reduction in Figures 11 to 17 was performed with d=1.7 and α=0.9. In all examples, noise reduction was performed appropriately.

[0138] Next, a learning program and a noise reduction program for executing the processing performed by the series of learning systems 10 and noise reduction systems 20 described above will be explained. As shown in Figure 18(a), the learning program 100 is stored in a program storage area 111 formed on a computer-readable recording medium 110 that is inserted into and accessed by a computer, or is provided by a computer. The recording medium 110 may be a non-temporary recording medium.

[0139] The learning program 100 comprises a learning acquisition module 101, a noise reduction module 102, a noise addition module 103, and a training module 104. The functions realized by executing the learning acquisition module 101, the noise reduction module 102, the noise addition module 103, and the training module 104 are the same as the functions of the learning acquisition unit 11, the noise reduction unit 12, the noise addition unit 13, and the training unit 14 of the learning system 10 described above.

[0140] As shown in Figure 18(b), the noise reduction program 200 is stored in a program storage area 211 formed on a computer-readable recording medium 210 that is inserted into and accessed by the computer, or is provided by the computer. The recording medium 210 may be a non-temporary recording medium. If the learning program 100 and the noise reduction program 200 are executed on the same computer, the recording medium 210 may be the same as the recording medium 110.

[0141] The noise reduction program 200 comprises a noise reduction acquisition module 201 and a noise reduction module 202. The functions realized by running the noise reduction acquisition module 201 and the noise reduction module 202 are the same as the functions of the noise reduction acquisition unit 21 and the noise reduction unit 22 of the noise reduction system 20 described above.

[0142] Furthermore, the learning program 100 and the noise reduction program 200 may be configured such that part or all of them are transmitted via a transmission medium such as a communication line, received by other equipment, and recorded (including installation). Also, each module of the learning program 100 and the noise reduction program 200 may be installed on any of multiple computers, not just one. In that case, the series of processes described above will be performed by a computer system consisting of these multiple computers.

[0143] The learning method, inference model, denoising method, denoising program, and denoising system of this disclosure have the following configurations. [1] A learning method for generating an inference model used to remove noise from data having multiple values, A training data acquisition step involves acquiring training data that has multiple values ​​and contains noise, and generating power-exponential training data in which each of the multiple values ​​in the acquired training data has a value raised to a power of a predetermined value, and A noise reduction step is performed to generate denoised training data from the power-expanded training data generated in the training acquisition step using an inference model in the middle of training, A noise addition step is performed to generate noise-added training data by adding noise based on a predetermined noise to the training data after noise removal generated in the noise removal step, The training step includes training the machine learning of an inference model using a combination of the power-exponentiated training data generated in the training acquisition step and the noise-added training data generated in the noise-adding step, each of which is used as data containing noise. After the inference model is trained in the training step, in the denoising step, the trained inference model is used as an in-progress inference model to generate denoised training data. A learning method comprising repeating the noise reduction step, the noise addition step, and the training step. [2] The learning method according to [1], wherein in the learning acquisition step, a value is used as the pre-set value such that there is a linear relationship between the value of the powered-up learning data and the magnitude of the variability of said value. [3] The learning method according to [1] or [2], wherein in the noise addition step, weighting is performed for each of the preset noise values ​​according to the values ​​of the training data after noise removal, and weighted noise is added to generate noise-added training data. [4] The learning method described in [3], wherein in the noise addition step, the values ​​α and β set in advance for weighting such that 0.0 < α ≤ 1.0 and -1.0 < β < 1.0 are weighted by α × the value of the training data after noise removal + β. [5] A learning method according to any one of [1] to [4], wherein each time the inference model in the training step is trained, the denoising step generates denoised training data using the trained inference model as an inference model in training. [6] A learning method according to any one of [1] to [5], wherein in the training step, in any one of the iterations, the machine learning of the inference model is trained using the power-up training data generated in the training acquisition step instead of the noise-added training data generated in the noise-added step. [7] The learning method according to any one of [1] to [6], wherein in the noise addition step, the noise is set according to the repetitions and is set according to the repetitions. [8] The learning method according to any one of [1] to [7], wherein in the noise addition step, the noise is of the same type as the noise assumed to be removed, as the pre-set noise. [9] An inference model for causing a computer to take data having multiple values ​​as input and perform operations according to the input to output information, An inference model generated by one of the learning methods described in [1] to [8].

[10] A denoising method for removing noise from data having multiple values, using an inference model used to remove noise from data having multiple values, A removal acquisition step involves acquiring noise-removal target data that has multiple values ​​and is subject to noise removal, and generating power-removal target data for each of the multiple values ​​in the acquired noise-removal target data, where each value is raised to a power of a predetermined value. A noise reduction step is to generate noise-reduced data from the power-expanded noise-reduced data generated in the removal acquisition step using the inference model, and to generate noise-reduced result data of the noise-reduced data having a value obtained by raising each of the multiple values ​​of the generated noise-reduced data to the power of the reciprocal of the preset value, A noise reduction method that includes [details omitted].

[11] The noise reduction method according to

[10] , wherein in the removal acquisition step, a value is used as the preset value such that the value of the power-expanded noise removal target data and the magnitude of the variation of said value have a linear relationship.

[12] The noise reduction method according to

[10] or

[11] , wherein in the removal acquisition step, an interference image by coherent waves is acquired as data to be denoised.

[13] Further includes an OCT step of acquiring an optical interference image by OCT, A noise reduction method according to any one of

[10] to

[13] , wherein in the removal acquisition step, the optical interference image acquired in the OCT step is acquired as data to be denoised.

[14] The inference model is generated by the learning method described in any of [1] to [8], and the noise reduction method described in any of

[10] to

[13] .

[15] A denoising program that causes a computer to operate as a denoising system that removes noise from data having multiple values ​​using an inference model used to remove noise from data having multiple values, The aforementioned computer, A removal acquisition means that acquires noise removal target data which has multiple values ​​and is subject to noise removal, and generates power-removed noise removal target data which has a value obtained by raising each of the multiple values ​​in the acquired noise removal target data to a power of a predetermined value, A noise reduction means generates noise-reduced data from the power-expanded noise-reduced data generated by the removal acquisition means using the inference model, and generates noise-reduced result data of the noise-reduced data, wherein each of the multiple values ​​in the generated noise-reduced data has a value obtained by raising it to the power of the reciprocal of the preset value. A noise reduction program that functions as such.

[16] A denoising system for removing noise from data having multiple values, using an inference model used to remove noise from data having multiple values, A removal acquisition means that acquires noise removal target data which has multiple values ​​and is subject to noise removal, and generates power-removed noise removal target data which has a value obtained by raising each of the multiple values ​​in the acquired noise removal target data to a power of a predetermined value, A noise reduction means generates noise-reduced data from the power-expanded noise-reduced data generated by the removal acquisition means using the inference model, and generates noise-reduced result data of the noise-reduced data, wherein each of the multiple values ​​in the generated noise-reduced data has a value obtained by raising it to the power of the reciprocal of the preset value. A noise reduction system equipped with the following features.

[17] The OCT device further includes an OCT system that acquires optical interference images by OCT, The noise reduction system described in

[16] , wherein the removal acquisition means acquires an optical interference image acquired by an OCT device as data to be removed from noise. [Explanation of Symbols]

[0144] 10...Learning system, 11...Learning acquisition unit, 12...Noise reduction unit, 13...Noise addition unit, 14...Training unit, 20...Noise reduction system, 21...Removal acquisition unit, 22...Noise reduction unit, 100...Learning program, 101...Learning acquisition module, 102...Noise reduction module, 103...Noise addition module, 104...Training module, 110...Recording medium, 111...Program storage area, 200...Noise reduction program, 201...Removal acquisition module, 202...Noise reduction module, 210...Recording medium, 211...Program storage area.

Claims

1. A learning method for generating an inference model used to remove noise from data having multiple values, A training data acquisition step involves acquiring training data that has multiple values ​​and contains noise, and generating power-exponential training data in which each of the multiple values ​​in the acquired training data has a value raised to a power of a predetermined value, and A noise reduction step is performed to generate denoised training data from the power-law training data generated in the training acquisition step using an inference model in the middle of training, A noise addition step is performed to generate noise-added training data by adding noise based on a predetermined noise to the training data after noise removal generated in the noise removal step, The training step includes training the machine learning of an inference model using a combination of the power-exponential training data generated in the training acquisition step and the noise-added training data generated in the noise-adding step, each of which is used as data containing noise. After the inference model is trained in the training step, in the denoising step, the trained inference model is used as an in-progress inference model to generate denoised training data. A learning method comprising repeating the noise reduction step, the noise addition step, and the training step.

2. The learning method according to claim 1, wherein in the learning acquisition step, a value is used as the pre-set value such that there is a linear relationship between the value of the powered-up learning data and the magnitude of the variability of said value.

3. The learning method according to claim 1 or 2, wherein in the noise addition step, weighting is performed for each of the preset noise values ​​according to the values ​​of the training data after noise removal, and weighted noise is added to generate noise-added training data.

4. The learning method according to claim 3, wherein in the noise addition step, the weights α and β, which are predetermined values ​​for weighting such that 0.0 < α ≤ 1.0 and -1.0 < β < 1.0, are weighted by α × the value of the training data after noise removal + the value of β.

5. The learning method according to claim 1 or 2, wherein each time the inference model in the training step is trained, the denoising step generates denoised training data using the trained inference model as an inference model in training.

6. The learning method according to claim 1 or 2, wherein in one of the iterations, in the training step, the machine learning of the inference model is trained using the power-up-training data generated in the training acquisition step instead of the noise-added training data generated in the noise-added step.

7. The learning method according to claim 1 or 2, wherein in the noise addition step, the noise is set in advance according to the repetition.

8. The learning method according to claim 1 or 2, wherein in the noise addition step, the same type of noise as the noise to be removed is used as the pre-set noise.

9. An inference model for causing a computer to function by taking data with multiple values ​​as input, performing calculations according to the input, and outputting information, An inference model generated by the learning method described in claim 1 or 2.

10. A noise reduction method for removing noise from data having multiple values, using an inference model used to remove noise from data having multiple values, A removal acquisition step involves acquiring noise-removal target data that has multiple values ​​and is subject to noise removal, and generating power-removal target data for each of the multiple values ​​in the acquired noise-removal target data, where each value is raised to a power of a predetermined value. A noise reduction step is to generate noise-reduced data from the power-expanded noise-reduced data generated in the removal acquisition step using the inference model, and to generate noise-reduced result data of the noise-reduced data having a value obtained by raising each of the multiple values ​​of the generated noise-reduced data to the power of the reciprocal of the preset value, Includes, A noise reduction method in which, in the aforementioned removal acquisition step, a value is used as the pre-set value such that the value of the data to be noise-reduced after exponentiation has a linear relationship with the magnitude of the variation of said value.

11. The noise reduction method according to claim 10, wherein in the removal acquisition step, an interference image by a coherent wave is acquired as data to be denoised.

12. The method further includes an OCT step of acquiring an optical interference image by OCT, The noise reduction method according to claim 10, wherein in the removal acquisition step, the optical interference image acquired in the OCT step is acquired as data to be noise-reduced.

13. The inference model is generated by the learning method described in claim 1 or 2.

14. A noise reduction program that causes a computer to operate as a noise reduction system that removes noise from data having multiple values ​​using an inference model used to remove noise from data having multiple values, The aforementioned computer, A removal acquisition means that acquires noise removal target data which has multiple values ​​and is subject to noise removal, and generates power-removed noise removal target data which has a value obtained by raising each of the multiple values ​​in the acquired noise removal target data to a power of a predetermined value, A noise reduction means generates noise-reduced data from the power-expanded noise-reduced data generated by the removal acquisition means using the inference model, and generates noise-reduced result data of the noise-reduced data, wherein each of the multiple values ​​in the generated noise-reduced data has a value obtained by raising it to the power of the reciprocal of the preset value. To make it function as, The aforementioned removal acquisition means is a noise reduction program that uses, as the preset value, a value such that there is a linear relationship between the value of the data to be noise-reduced after exponentiation and the magnitude of the variation of said value.

15. A noise reduction system that removes noise from data having multiple values ​​using an inference model used to remove noise from data having multiple values, A removal acquisition means that acquires noise removal target data which has multiple values ​​and is subject to noise removal, and generates power-removed noise removal target data which has a value obtained by raising each of the multiple values ​​in the acquired noise removal target data to a power of a predetermined value, A noise reduction means generates noise-reduced data from the power-expanded noise-reduced data generated by the removal acquisition means using the inference model, and generates noise-reduced result data of the noise-reduced data, wherein each of the multiple values ​​in the generated noise-reduced data has a value obtained by raising it to the power of the reciprocal of the preset value. Equipped with, The noise removal acquisition means is a noise removal system that uses a value as the preset value such that the value of the power-expanded noise removal target data and the magnitude of the variation of said value have a linear relationship.

16. The system further includes an OCT device that acquires optical interference images using OCT. The noise reduction system according to claim 15, wherein the removal acquisition means acquires an optical interference image acquired by an OCT device as data to be removed from noise.

Images