Image processing device and image processing method

Abstract

This image processing device 1 comprises a CNN processing unit 11, a CNN training unit 12, a map creation unit 14, and an image mixing unit 15. In each of M times of processing, the CNN processing unit 11 inputs an input image z to a CNN and causes the CNN to output an output image. The CNN training unit 12 trains the CNN by using an evaluation function including an error evaluation term representing an evaluation value pertaining to an error between an output image x m output from the CNN in m-th processing of the CNN processing unit 11 and a target image x 0 . The map creation unit 14 creates an uncertainty map representing an uncertainty distribution of a pixel value of each pixel of the output image x M output in the M-th processing of the CNN processing unit 11. The image mixing unit 15 creates an image after noise reduction processing by using the object image x 0 , the output image x M and the uncertainty map. Thereby, an image processing device is implemented which can suppress a deterioration in the image quality due to excessive smoothing occurring in the noise reduction processing using DIP technology.

Description

Image processing device and image processing method

[0001] The present disclosure relates to an apparatus and method for performing noise reduction processing on a target image.

[0002] Various techniques are known for performing processing to reduce noise in images containing noise. Among these noise reduction processing techniques, a technique that utilizes Deep Image Prior technology, which uses a convolutional neural network, a type of deep neural network, has attracted attention. Hereinafter, Convolutional Neural Network will be referred to as "CNN," and Deep Image Prior technology will be referred to as "DIP technology."

[0003] The DIP technique utilizes the property of CNN that meaningful structures in an image are learned faster than random noise (i.e., random noise is difficult to learn).The DIP technique can reduce noise in the target image.

[0004] For example, tomographic images of a subject acquired by radiation tomography devices such as PET (Positron Emission Tomography) devices and SPECT (Single Photon Emission Computed Tomography) devices contain a lot of noise, and therefore require noise reduction processing. The invention disclosed in Patent Document 1 uses a tomographic image of the subject reconstructed based on coincidence counting information collected by the PET device as a target image, and creates a tomographic image after noise reduction processing using DIP technology.

[0005] Japanese Patent Application Laid-Open No. 2020-128882

[0006] J. Nuyts et al., "A concave prior penalizing relative differences for maximum-a-posteriori reconstruction in emission tomography", IEEE TNS, Vol.49, Issue 1, pp.56-60, 2002 Hiroyuki Kudo, "Image reconstruction method in low-exposure CT - Fundamentals of statistical image reconstruction, iterative image reconstruction, and compressed sensing", Medical Imaging Technology, Vol.32, No.4, pp.239-248, 2014Antonin Chambolle, "An Algorithm for Total Variation Minimization and Applications", Journal of Mathematical Imaging and Vision 20, pp.89-97, 2004K. Zou, Z. Chen, X. Yuan, X. Shen, M. Wang, & H. Fu, "A review of uncertainty estimation and its application in medical imaging", Meta-Radiology, Vol.1, 100003, 2023

[0007] Although noise reduction processing using conventional DIP technology has excellent noise reduction performance, it tends to excessively smooth the image, which can result in a decrease in quantitative accuracy and can cause discomfort in image quality when used in actual clinical settings. Other noise reduction processing technologies have similar problems.

[0008] An object of the present invention is to provide an image processing device and an image processing method that can suppress image quality degradation caused by excessive smoothing that occurs in noise reduction processing using DIP technology.

[0009] An embodiment of the present invention is an image processing device. ０The noise reduction device includes: (1) a CNN processing unit that inputs an input image to a convolutional neural network in each of M (M is an integer equal to or greater than 2) processes and outputs an output image from the convolutional neural network; and (2) a CNN processing unit that, between the m-th process and the (m+1)-th process of the M processes of the CNN processing unit, compares the output image output from the convolutional neural network in the m-th process of the CNN processing unit with a target image x ０ (3) a CNN learning unit that uses an evaluation function including an error evaluation term that represents an evaluation value regarding the error between the output image x outputted in the Mth processing of the CNN processing unit and the output image x Ｍ (4) a map generator for generating an uncertainty map representing the distribution of uncertainty in the pixel values ​​of each pixel in the target image x ０ , output image x Ｍ and the uncertainty map is used to calculate, for each pixel, the target image x ０ The weighting of the pixel value of ０ and the output image x Ｍ The weighting of the pixel value of Ｍ Let the magnitude of uncertainty of pixel values ​​in the uncertainty map be σ, then the ratio (w ０ / w Ｍ ) is expressed as an increasing function with σ as a variable. ０ , w Ｍ and an image mixer that creates a noise-reduced image based on the weighted average of pixel values ​​obtained by:

[0010] An embodiment of the present invention is an image processing method. ０ The method for performing noise reduction processing on a target image x includes: (1) a CNN processing step in which an input image is input to a convolutional neural network in each of M processing times (M is an integer of 2 or more), and an output image is output from the convolutional neural network; and (2) between the m-th processing and the (m+1)-th processing of the M processing times of the CNN processing step, a comparison is made between the output image output from the convolutional neural network in the m-th processing of the CNN processing step and a target image x ０(3) a CNN learning step in which a convolutional neural network is trained based on the value of an evaluation function including an error evaluation term that represents an evaluation value regarding the error between the output image x Ｍ (4) a map generation step of generating an uncertainty map representing the distribution of uncertainty in the pixel values ​​of each pixel in the target image x ０ , output image x Ｍ and the uncertainty map is used to calculate, for each pixel, the target image x ０ The weighting of the pixel value of ０ and the output image x Ｍ The weighting of the pixel value of Ｍ Let the magnitude of uncertainty of pixel values ​​in the uncertainty map be σ, then the ratio (w ０ / w Ｍ ) is expressed as an increasing function with σ as a variable. ０ , w Ｍ and an image blending step of creating a noise-reduced image based on a weighted average of pixel values ​​obtained by:

[0011] According to an embodiment of the present invention, it is possible to suppress deterioration in image quality due to excessive smoothing that occurs in noise reduction processing using the DIP technique.

[0012] FIG. 1 is a diagram showing the configuration of an image processing device 1. FIG. 2 is a flowchart of an image processing method. FIG. 3 is a diagram showing an example of the configuration of a CNN. FIG. 4 is a diagram explaining adjacent pixels in an output image. FIG. 5 is a diagram showing (a) a target image x ０ (b) a target image x ０ 6A and 6B are diagrams showing an image obtained by noise reduction processing using a Gaussian filter on a target image x as a comparative example 1. ０ On the other hand, the output image x obtained by noise reduction processing using the conventional DIP technology Ｍ (b) a target image x ０ 7A and 7B are diagrams showing an image obtained by the noise reduction process of this embodiment, respectively.０ Figure 8 is a graph comparing PVR at the location of nuclei in each image for eight subjects. Figure 9 is a graph comparing standard deviations of pixel values ​​in white matter regions in each image for eight subjects.

[0013] Hereinafter, embodiments of an image processing device and an image processing method will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are designated by the same reference numerals, and duplicate explanations will be omitted. The present invention is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope of the claims.

[0014] FIG. 1 is a diagram showing the configuration of an image processing device 1. The image processing device 1 processes a target image x ０ This involves performing noise reduction processing on the image.

[0015] The image processing device 1 includes a GPU (Graphics Processing Unit) that performs processing using a convolutional neural network (CNN), an input unit (e.g., a keyboard or mouse) that accepts input from an operator, a display unit (e.g., a liquid crystal display) that displays images, etc., and a storage unit that stores programs and data for executing various processes. The image processing device 1 is a computer having a CPU, RAM, ROM, hard disk drive, etc.

[0016] The image processing device 1 includes a CNN processing unit 11, a CNN learning unit 12, a storage unit 13, a map creation unit 14, and an image mixing unit 15. ０ The image processing method for performing noise reduction processing on an image includes a CNN processing step, a CNN learning step, a map creation step, and an image blending step.

[0017] The CNN processing unit 11 inputs an input image z to the CNN in each of M (M is an integer of 2 or more) processings and outputs an output image from the CNN (CNN processing step). ｍIt is expressed as:

[0018] The CNN learning unit 12 calculates the output image x output from the CNN in the mth process of the CNN processing unit 11 between the mth process and the (m+1)th process of the Mth process of the CNN processing unit 11. ｍ and the target image x ０ Using an evaluation function including an error evaluation term that represents an evaluation value regarding the error between m and m, the CNN is trained based on the value of this evaluation function (CNN training step). Here, m is an integer between 1 and (M-1).

[0019] The processes of the CNN processing unit 11 and the CNN learning unit 12 are alternately repeated multiple times. In the conventional DIP technique, the output image x Ｍ is the image after noise reduction processing.

[0020] Target image x to be subjected to noise reduction processing ０ can be any image. The input image z can also be any image. The input image z can be a random noise image.

[0021] A tomographic image of a subject acquired by a radiation tomography apparatus (e.g., a PET apparatus) is referred to as a target image x. ０ In this case, the target image x ０ may be an image reconstructed by any image reconstruction method (e.g., FBP, MLEM, OSEM, DRAMA, etc.) based on coincidence counting information. ０ When the input image z is a tomographic image of the subject, the input image z may be an image representing morphological information of the subject, or may be an MRI image, a CT image, or a static PET image of the subject.

[0022] Target image x ０ may be a two-dimensional image or a three-dimensional image. ０ is a two-dimensional image, the input image z and the output image x ｍ is also a two-dimensional image. ０ is a three-dimensional image, the input image z and the output image x ｍ is also a three-dimensional image.

[0023] The storage unit 13 stores the output image x output from the CNN processing unit 11. １ ~x Ｍ Among these, the output images required for the processes of the map creating unit 14 and the image mixing unit 15 are stored.

[0024] The map creation unit 14 calculates the output image x output in the Mth processing of the CNN processing unit 11. Ｍ The image blending unit 15 creates an uncertainty map that represents the distribution of uncertainty in the pixel values ​​of the target image x ０ , output image x Ｍ The process of the map creating unit 14 and the image blending unit 15 will be described in detail later.

[0025] 2 is a flowchart of the image processing method. In step S1, m=1. In the subsequent CNN processing step S2, an input image z is input to the CNN by the mth processing of the CNN processing unit 11, and an output image x is output from the CNN. ｍ will be output.

[0026] In the following step S3, it is determined whether m has reached M. If it is determined in step S3 that m has not reached M, the process proceeds to CNN learning step S4. If it is determined in step S3 that m has reached M, the process proceeds to map creation step S6.

[0027] If it is determined in step S3 that m has not reached M, the CNN learning unit 12 learns the CNN in CNN learning step S4, and then in the following step S5, m is incremented by 1, and then the process returns to CNN processing step S2. As a result, the CNN processing step S2 and the CNN learning step S4 are alternately repeated until m reaches M.

[0028] If it is determined in step S3 that m has reached M, an uncertainty map is created by the map creation unit 14 in map creation step S6, and then an image after noise reduction processing is created by the image blending unit 15 in the subsequent image blending step S7.

[0029] Figure 3 shows an example of a CNN configuration. The CNN shown in this figure has a three-dimensional U-net structure including an encoder and a decoder. This figure shows the spatial size of each layer of the CNN, assuming that the number of pixels of the input image z input to the CNN is NxNx64.

[0030] Next, the evaluation function E used by the CNN learning unit 12 in the CNN learning step S4 will be described. The processing by the CNN is represented by f, and the weighting coefficient parameter representing the learning state of the CNN is represented by θ. θ changes as the learning of the CNN progresses. The weighting coefficient for the mth processing of the CNN processing unit 11 is represented by θ. ｍ When an input image z is input to this CNN, the output image x is ｍ is expressed by the following formula (1).

[0031] The evaluation function E may be any function, such as the L1 norm, the L2 norm, or the negative logarithmic likelihood in the Poisson distribution. Here, however, it is expressed as the mean squared error (MSE) expressed by the following formula (2). Since the number of data in the DIP technique is 1, the MSE is equal to the squared error (SE). This evaluation function E is calculated by ｍ and the target image x ０ It contains only an error estimate that represents an estimate of the error between

[0032] The evaluation function E shown in the following equation (3) includes not only an error evaluation term (first term on the right-hand side) but also a regularization term (second term on the right-hand side) for suppressing CNN overlearning. This regularization term is ｍ This regularization term represents an evaluation value for the difference in pixel values ​​between adjacent pixels in the output image x ｍ β is a hyperparameter that adjusts the degree of regularization effect. The smaller β is, the smaller the effect of regularization is. The larger β is, the greater the effect of regularization (i.e., the effect of suppressing CNN overfitting).

[0033] In the case of a two-dimensional image, the neighboring pixels of a pixel include neighboring pixels in each of two mutually perpendicular directions, and preferably also neighboring pixels in a diagonal direction. In the case of a two-dimensional image, the number of neighboring pixels of a pixel is eight, excluding pixels located at the edges or corners of the image.

[0034] In the case of a three-dimensional image, the neighbors of a pixel include neighbors in each of three mutually orthogonal directions, and preferably also neighbors in diagonal directions. In the case of a three-dimensional image, the number of neighbors of a pixel is 26, excluding pixels located at the edges or corners of the image.

[0035] 4 is a diagram illustrating adjacent pixels in an output image. This diagram shows the output image as a two-dimensional image, with 3×3 pixels. The pixel value of the pixel at the center of this diagram is λ ｊ The pixel values ​​of the eight pixels adjacent to this central pixel are set as λ ｋ (k=1 to 8), the difference in pixel value between adjacent pixels with respect to this central pixel is |λ ｊ -λ ｋ The regularization term represents an evaluation value for the difference in pixel values ​​for all combinations of adjacent pixels in the output image.

[0036] The regularization term may be expressed by various formulas as long as it represents an evaluation value relating to the difference in pixel values ​​between adjacent pixels in the output image. For example, the regularization term is expressed by the following formula (4). In this formula (4), N ｊ represents the set of pixels k adjacent to pixel j. γ represents the pixel value λ ｊ This equation (4) includes a term for the difference in pixel values ​​of adjacent pixels in the numerator and a term for the sum of pixel values ​​of adjacent pixels in the denominator, and represents an evaluation value related to the relative difference in pixel values ​​between adjacent pixels in the output image.

[0037] It should be noted that equation (4) is similar to the equation described in Non-Patent Document 1. However, in Non-Patent Document 1, the equation similar to equation (4) is used in the process of reconstructing a tomographic image of a subject based on coincidence counting information collected by a PET device, and is not used in performing noise reduction processing on the tomographic image by the DIP technique.

[0038] Furthermore, as the regularization term, for example, Gibbs prior (Non-Patent Document 2) or total variation (Non-Patent Document 3) may be used. Note that these documents also describe a technique for reconstructing a tomographic image of a subject, but do not describe a technique for performing noise reduction processing on a tomographic image using the DIP technique.

[0039] Next, the details of the process of the map creation unit 14 will be described. The map creation unit 14 creates an output image x Ｍ An uncertainty map is created that represents the distribution of uncertainty in the pixel value of each pixel in

[0040] In the process of alternately repeating the processing of the CNN processing unit 11 and the CNN learning unit 12 multiple times, the output image x ｍ As m increases, the pixel values ​​of the pixels in the final output image x may not converge to a certain value, but may increase and decrease repeatedly and become unstable. Ｍ The pixel value of the pixel in question at is highly uncertain. The uncertainty map represents the distribution of such instability or uncertainty in pixel values.

[0041] The map creation unit 14 can create the uncertainty map using various methods. For example, the map creation unit 14 can create the uncertainty map based on output images output from each of N (N is an integer equal to or greater than 2) processes out of M processes by the CNN processing unit 11. In this case, it is preferable that the map creation unit 14 creates the uncertainty map based on output images output from each of N processes closest to the Mth process, which is the final process out of M processes by the CNN processing unit 11.

[0042] The map creating unit 14 may create, as an uncertainty map, a map that represents the distribution of the difference between the maximum pixel value and the minimum pixel value of each pixel, based on the output images output in each of the N processes.

[0043] Preferably, as expressed in the following equation (5), the map creation unit 14 may create a map σ representing the distribution of the standard deviation of the pixel values ​​of each pixel as an uncertainty map based on the output images output in each of the N processing steps.

[0044] σ ｊ is the value of the j-th pixel in the uncertainty map σ, and ｍ is the standard deviation of the pixel value of the j-th pixel in ｊ is the N output images x ｍ is the average value of the j-th pixel in the output image x M-cycle×n The cycle is the pixel value of the j-th pixel in the N output images x ｍ The value of this parameter cycle may be 1, but is set to an appropriate value taking into consideration the speed at which pixel values ​​repeatedly increase and decrease as m increases. The values ​​of N and cycle may also be determined empirically.

[0045] The uncertainty map may also be one described in Non-Patent Document 4. For example, the Ensemble Methods described in Non-Patent Document 4 can create multiple networks using the same data and calculate uncertainty based on the networks. Furthermore, the Bayesian Neural Networks described in Non-Patent Document 4 can approximately sample several models by training some networks with the weights set to zero during training. In addition, Non-Patent Document 4 describes methods such as Test-time Data Augmentation and Deterministic Methods.

[0046] Next, the details of the processing of the image mixing unit 15 will be described.０ , output image x Ｍ and the uncertainty map σ to create a noise-reduced image.

[0047] Specifically, the image blending unit 15 blends the target image x ０ The weighting of the pixel value of ０ and the output image x Ｍ The weighting of the pixel value of Ｍ Let the magnitude of uncertainty of pixel values ​​in the uncertainty map be σ, then the ratio (w ０ / w Ｍ ) is expressed as an increasing function with σ as a variable. ０ , w Ｍ The noise-reduced image is created based on the weighted average of the pixel values ​​calculated by the ratio (w ０ / w Ｍ ) can take values ​​ranging from 0 to infinity. Here, the increasing function does not have to be one that can be expressed by an equation. An increasing function g is a function such that g(a)≦g(b) holds when a<b.

[0048] For example, weighting w ０ , w Ｍ may be expressed by the following formula (7): ０,ｊ is the target image x ０ is the weighting for the pixel value of the j-th pixel in Ｍ,ｊ is the output image x Ｍ is the weighting of the pixel value of the j-th pixel in σ mean is the average pixel value of all pixels in the uncertainty map. α is a positive coefficient. The image blending unit 15 uses the weighting w ０ , w Ｍ Based on the weighted average of pixel values ​​obtained by the above, a noise-reduced image x is created (the following equation (8)).

[0049] In the above equation (7), the value of α may be variable. ０ / w ＭThe relationship between the magnitude σ of uncertainty of pixel values ​​in the uncertainty map and the ratio (w ０ / w Ｍ ) is preferably variable and the relationship can be selected by the user.

[0050] For example, the following configuration may be used in the above formula (7) in which the α value is variable and can be set by the user.

[0051] A graphical input interface for setting the α value may be displayed on the display unit of the image processing device 1, and the α value may be set by the user operating a mouse on the input interface. Alternatively, the α value may be set by the user entering data from a keyboard. Alternatively, the image x after noise reduction processing for the set α value may be immediately displayed on the display unit, allowing the user to reset the α value while checking the image x.

[0052] Next, an example will be described. In this example, a head-dedicated PET device Vrain was used to measure the amount of a drug [ 18 Coincidence counting information was collected for each of the eight subjects' heads to which F]FDG was administered, and a tomographic image reconstructed based on the collected coincidence counting information was taken as the target image x ０ The MRI image of the subject was used as the input image z. The above formula (2) was used as the evaluation function E. The uncertainty map σ was calculated using the above formula (5) with M=200, N=20, and cycle=5. The weighting w ０ , w Ｍ was calculated using the above formula (7).

[0053] 5 to 7 show various images of a specific subject among the eight subjects. ０ This target image x ０ The dark area indicated by the two arrows in the central region of the image is the nerve nucleus where the drug is highly concentrated. ０In contrast, an image obtained by noise reduction processing using a Gaussian filter is shown as Comparative Example 1.

[0054] FIG. 6(a) shows the target image x ０ On the other hand, the output image x obtained by noise reduction processing using the conventional DIP technology Ｍ is shown as Comparative Example 2. ０ 7 shows the image x obtained by the noise reduction process of this embodiment using the above equation (8). Fig. 7(a) shows the uncertainty map obtained in the process of this embodiment. Fig. 7(b) shows the weighting w obtained in the process of this embodiment. ０ A map of the following is shown.

[0055] The image of Comparative Example 2 shown in FIG. 6(a) (the output image x obtained by noise reduction processing using the conventional DIP technology) Ｍ ), the image of this embodiment shown in FIG. 6B (the image x after noise reduction processing obtained by the above formula (8)) has a noise reduction effect on the target image x shown in FIG. ０ In contrast, noise is effectively removed and the decrease in quantitative accuracy is suppressed in areas where the drug is highly concentrated (for example, in the area of ​​a nerve nucleus).

[0056] 8 is a graph comparing the peak-to-valley ratio (PVR) at the nucleus region in each image of eight subjects. As shown in this graph, the PVR value at the nucleus region in the image of Comparative Example 2 is smaller than that in the image of Comparative Example 1. However, the PVR value at the nucleus region in the image of this example is larger than that in either of the images of Comparative Examples 1 or 2.

[0057] Figure 9 is a graph comparing the standard deviations of pixel values ​​in white matter regions in images of eight subjects. Because white matter in the brain is a homogeneous region, the standard deviation of pixel values ​​in white matter regions is inherently small. As shown in this graph, the image in this example has the smallest standard deviation of pixel values ​​in the white matter region.

[0058] As described above, according to this embodiment, the target image x ０It is possible to effectively remove noise while maintaining quantitativeness, and to suppress image quality degradation caused by excessive smoothing that occurs in noise reduction processing using the DIP technique.

[0059] The image processing device and image processing method are not limited to the above-described embodiment and configuration example, and various modifications are possible.

[0060] The image processing device of the first aspect according to the above embodiment is ０ The noise reduction device includes: (1) a CNN processing unit that inputs an input image to a convolutional neural network in each of M (M is an integer equal to or greater than 2) processes and outputs an output image from the convolutional neural network; and (2) a CNN processing unit that, between the m-th process and the (m+1)-th process of the M processes of the CNN processing unit, compares the output image output from the convolutional neural network in the m-th process of the CNN processing unit with a target image x ０ (3) a CNN learning unit that uses an evaluation function including an error evaluation term that represents an evaluation value regarding the error between the output image x outputted in the Mth processing of the CNN processing unit and the output image x Ｍ (4) a map generator for generating an uncertainty map representing the distribution of uncertainty in the pixel values ​​of each pixel in the target image x ０ , output image x Ｍ and the uncertainty map is used to calculate, for each pixel, the target image x ０ The weighting of the pixel value of ０ and the output image x Ｍ The weighting of the pixel value of Ｍ Let the magnitude of uncertainty of pixel values ​​in the uncertainty map be σ, then the ratio (w ０ / w Ｍ ) is expressed as an increasing function with σ as a variable. ０ , w Ｍ and an image mixer that creates a noise-reduced image based on the weighted average of pixel values ​​obtained by:

[0061] In the image processing device of the second aspect, in the configuration of the first aspect, the map creation unit may be configured to create an uncertainty map based on the output images output in each of N (N is an integer greater than or equal to 2) of the M processes of the CNN processing unit.

[0062] In the image processing device of the third aspect, in the configuration of the second aspect, the map creation unit may be configured to create a map representing the distribution of standard deviations of pixel values ​​of each pixel as an uncertainty map based on the output images output by each of N processing steps of the CNN processing unit.

[0063] In the image processing device of the fourth aspect, in the configuration of any one of the first to third aspects, the image mixing unit calculates the magnitude σ of uncertainty of pixel values ​​in the uncertainty map and the ratio (w ０ / w Ｍ ) may be configured to be variable.

[0064] In the image processing device of the fifth aspect, in the configuration of any one of the first to fourth aspects, the target image x ０ may be a tomographic image of the subject created based on coincidence counting information collected by a radiation tomography apparatus.

[0065] In the image processing device of a sixth aspect, in the configuration of the fifth aspect, the CNN processing unit may be configured to input an image representing morphological information of the subject as an input image to the convolutional neural network.

[0066] In the image processing device of a seventh aspect, in the configuration of the fifth aspect, the CNN processing unit may be configured to input an MRI image of the subject as an input image to the convolutional neural network.

[0067] In the image processing device of an eighth aspect, in the configuration of the fifth aspect, the CNN processing unit may be configured to input a CT image of the subject as an input image to the convolutional neural network.

[0068] In the image processing device of a ninth aspect, in the configuration of the fifth aspect, the CNN processing unit may be configured to input a static PET image of the subject as an input image to the convolutional neural network.

[0069] In the image processing device of a tenth aspect, in the configuration of any one of the first to fifth aspects, the CNN processing unit may be configured to input a random noise image as an input image to the convolutional neural network.

[0070] The image processing method of the first aspect according to the above embodiment is ０ The method for performing noise reduction processing on a target image x includes: (1) a CNN processing step in which an input image is input to a convolutional neural network in each of M processing times (M is an integer of 2 or more), and an output image is output from the convolutional neural network; and (2) between the m-th processing and the (m+1)-th processing of the M processing times of the CNN processing step, a comparison is made between the output image output from the convolutional neural network in the m-th processing of the CNN processing step and a target image x ０ (3) a CNN learning step in which a convolutional neural network is trained based on the value of an evaluation function including an error evaluation term that represents an evaluation value regarding the error between the output image x Ｍ (4) a map generation step of generating an uncertainty map representing the distribution of uncertainty in the pixel values ​​of each pixel in the target image x ０ , output image x Ｍ and the uncertainty map is used to calculate, for each pixel, the target image x ０ The weighting of the pixel value of ０ and the output image x Ｍ The weighting of the pixel value of Ｍ Let the magnitude of uncertainty of pixel values ​​in the uncertainty map be σ, then the ratio (w ０ / w Ｍ ) is expressed as an increasing function with σ as a variable. ０ , w Ｍ and an image blending step of creating a noise-reduced image based on a weighted average of pixel values ​​obtained by:

[0071] In the image processing method of the second aspect, in the configuration of the first aspect, the map creation step may be configured to create an uncertainty map based on output images output in each of N (N is an integer greater than or equal to 2) of M processes in the CNN processing step.

[0072] In the image processing method of the third aspect, in the configuration of the second aspect, in the map creation step, a map representing the distribution of standard deviations of pixel values ​​of each pixel may be created as an uncertainty map based on the output images output in each of the N processing steps of the CNN processing step.

[0073] In the image processing method of the fourth aspect, in the configuration of any one of the first to third aspects, in the image blending step, the magnitude σ of uncertainty of pixel values ​​in the uncertainty map and the ratio (w ０ / w Ｍ ) may be configured to be variable.

[0074] In the image processing method of the fifth aspect, in the configuration of any one of the first to fourth aspects, ０ may be a tomographic image of the subject created based on coincidence counting information collected by a radiation tomography apparatus.

[0075] In the image processing method of the sixth aspect, in the configuration of the fifth aspect, the CNN processing step may be configured to input an image representing morphological information of the subject as an input image to a convolutional neural network.

[0076] In the image processing method of the seventh aspect, in the configuration of the fifth aspect, the CNN processing step may be configured to input an MRI image of the subject as an input image to a convolutional neural network.

[0077] In the image processing method of the eighth aspect, in the configuration of the fifth aspect, the CNN processing step may be configured to input a CT image of the subject as an input image to a convolutional neural network.

[0078] In the image processing method of the ninth aspect, in the configuration of the fifth aspect, the CNN processing step may be configured to input a static PET image of the subject as an input image to a convolutional neural network.

[0079] In the image processing method of the tenth aspect, in the configuration of any one of the first to fifth aspects, in the CNN processing step, a random noise image may be input to the convolutional neural network as an input image.

[0080] The present invention can be used as an image processing device and an image processing method that can suppress image quality degradation caused by excessive smoothing that occurs in noise reduction processing using DIP technology.

[0081] 1...image processing device, 11...CNN processing unit, 12...CNN learning unit, 13...storage unit, 14...map creation unit, 15...image mixing unit

Claims

1. Target image x ０ An apparatus for performing noise reduction processing on a target image, comprising: a CNN processing unit that inputs an input image to a convolutional neural network in each of M (M is an integer of 2 or more) processes and outputs an output image from the convolutional neural network; between the m-th process and the (m + 1)-th process among the M processes of the CNN processing unit, an error evaluation term representing an evaluation value related to the error between the output image output from the convolutional neural network in the m-th process of the CNN processing unit and the target image x ０ A CNN learning unit that uses an evaluation function including an error evaluation term representing an evaluation value related to the error between the output image output from the convolutional neural network in the m-th process of the CNN processing unit and the target image x, and learns the convolutional neural network based on the value of this evaluation function; an uncertainty map creating unit that creates an uncertainty map representing the distribution of the uncertainty of the pixel values of each pixel in the output image x Ｍ output in the M-th process of the CNN processing unit; using the target image x ０ , the output image x Ｍ and the uncertainty map, for each pixel, the weight for the pixel value of the target image x ０ is set as w ０ , the weight for the pixel value of the output image x Ｍ is set as w Ｍ , and when the magnitude of the uncertainty of the pixel value in the uncertainty map is σ, the weights w ０ / w Ｍ are set so that the ratio (w ０ , w Ｍ ) is represented by an increasing function with σ as a variable, and an image mixing unit that creates an image after noise reduction processing based on the weighted average of the pixel values obtained by the weights w , an image processing apparatus.

2. The image processing apparatus according to claim 1, wherein the map creation unit creates the uncertainty map based on the output images output in each of N (N is an integer of 2 or more) of the M times of processing by the CNN processing unit.

3. The image processing apparatus according to claim 2, wherein the map creation unit creates, as the uncertainty map, a map representing the distribution of the standard deviation of the pixel values of each pixel based on the output images output in each of the N times of processing by the CNN processing unit.

4. The image mixing unit has a variable relationship between the magnitude σ of the uncertainty of the pixel value in the uncertainty map and the ratio (w ０ / w Ｍ ). The image processing apparatus according to any one of claims 1 to 3.

5. The target image x ０ is a tomographic image of a subject created based on coincidence counting information collected by a radiation tomography apparatus, and the image processing apparatus according to any one of claims 1 to 4.

6. The image processing apparatus according to claim 5, wherein the CNN processing unit inputs an image representing the morphological information of the subject as the input image to the convolutional neural network.

7. The image processing apparatus according to claim 5, wherein the CNN processing unit inputs an MRI image of the subject as the input image to the convolutional neural network.

8. The image processing apparatus according to claim 5, wherein the CNN processing unit inputs a CT image of the subject as the input image to the convolutional neural network.

9. The image processing apparatus according to claim 5, wherein the CNN processing unit inputs a static PET image of the subject as the input image to the convolutional neural network.

10. The image processing apparatus according to any one of claims 1 to 5, wherein the CNN processing unit inputs a random noise image as the input image to the convolutional neural network.

11. A method for performing noise reduction processing on a target image x ０ The method includes: a CNN processing step of inputting an input image into a convolutional neural network in each of M (M is an integer of 2 or more) processes and outputting an output image from the convolutional neural network; between the m-th process and the (m + 1)-th process among the M processes of the CNN processing step, an evaluation function including an error evaluation term representing an evaluation value related to the error between the output image output from the convolutional neural network in the m-th process of the CNN processing step and the target image x ０ and using the evaluation function to train the convolutional neural network based on the value of this evaluation function; a map creation step of creating an uncertainty map representing the distribution of the uncertainty of the pixel values of each pixel in the output image x Ｍ output in the M-th process of the CNN processing step; using the target image x ０ , the output image x Ｍ and the uncertainty map, for each pixel, setting the weighting for the pixel value of the target image x ０ as w ０ , setting the weighting for the pixel value of the output image x Ｍ as w Ｍ , and when the magnitude of the uncertainty of the pixel value in the uncertainty map is σ, the weights w ０ / w Ｍ are set so that the ratio is represented by an increasing function with σ as a variable, and an image mixing step of creating an image after noise reduction processing based on the weighted average of the pixel values obtained by the weights w ０ , w Ｍ . An image processing method comprising the above steps.

12. The image processing method according to claim 11, wherein in the map creation step, the uncertainty map is created based on the output images output in each of N (N is an integer of 2 or more) of the M times of processing in the CNN processing step.

13. The image processing method according to claim 12, wherein in the map creation step, a map representing the distribution of the standard deviation of the pixel values of each pixel is created as the uncertainty map based on the output images output in each of the N times of processing in the CNN processing step.

14. In the image mixing step, the relationship between the magnitude of uncertainty σ of the pixel value in the uncertainty map and the ratio (w ０ / w Ｍ ) is variable. The image processing method according to any one of claims 11 to 13.

15. The target image x ０ is a tomographic image of a subject created based on simultaneous count information collected by a radiation tomography apparatus, and is the image processing method according to any one of claims 11 to 14.

16. The image processing method according to claim 15, wherein in the CNN processing step, an image representing the morphological information of the subject is input as the input image to the convolutional neural network.

17. The image processing method according to claim 15, wherein in the CNN processing step, an MRI image of the subject is input as the input image to the convolutional neural network.

18. The image processing method according to claim 15, wherein in the CNN processing step, a CT image of the subject is input into the convolutional neural network as the input image.

19. The image processing method according to claim 15, wherein in the CNN processing step, a static PET image of the subject is input into the convolutional neural network as the input image.

20. The image processing method according to any one of claims 11 to 15, wherein in the CNN processing step, a random noise image is input into the convolutional neural network as the input image.

Images