Image processing apparatus and image processing method

Abstract

An image processing device (1) is provided with an input image creation unit (10), a first calculation unit (20), and a second calculation unit (30), and creates a noise-reduced image by reducing the noise of a target image (52). The first calculation unit (20) includes a first CNN processing unit (21) and a first CNN learning unit (22), and performs processing for supervising pre-learning. The first input image (40) is an image in which the pixel value of a partial region is changed on the basis of the teacher image (42). The second calculation unit (30) includes a second CNN processing unit (31) and a second CNN learning unit (32), and performs unsupervised learning processing. As a result, an image processing device capable of easily reducing the noise of a target image by performing unsupervised learning after supervised advance learning on a CNN is achieved.

Description

Technical Field

[0001] The present invention relates to an image processing device and an image processing method for reducing noise of an object image to produce a noise-reduced image. Background Art

[0002] An image sometimes contains noise. An example of an image containing noise is a cross-sectional image of a subject reconstructed based on information obtained by a radiation tomography device. The radiation tomography device is a PET (Positron Emission Tomography) device and a SPECT (Single Photon Emission Computed Tomography) device.

[0003] The PET device has a detection unit having a plurality of small radiation detectors arranged around a measurement space where a subject is placed. The PET device detects photon pairs with an energy of 511 keV generated by the cancellation of electrons and positrons in a subject to which a positron emitting isotope (RI ray source) is administered using a coincidence counting method by the detection unit, and collects the coincidence counting information.

[0004] Then, based on the collected multiple simultaneous counting information, a tomographic image showing the spatial distribution of the generation frequency of photon pairs in the measurement space (i.e., the spatial distribution of RI ray sources) can be reconstructed. Such PET devices play an important role in the field of nuclear medicine, etc., and can be used to study, for example, biological functions or higher-level functions of the brain.

[0005] Since the reconstructed tomographic image of the subject contains a lot of statistical noise, it is required to reduce the noise of the tomographic image. In addition, not limited to such PET images (tomographic images), it is sometimes required to reduce the noise from a noisy object image to create a reduced-noise image.

[0006] As a noise reduction technology, various methods are known. Among them, the technology of reducing noise by using the Deep Image Prior technology of a convolutional neural network, which is a deep neural network, has attracted attention. Hereinafter, the convolutional neural network (Convolutional Neural Network) is referred to as "CNN" and the Deep Image Prior technology is referred to as "DIP technology".

[0007] In the DIP technology, the property of CNN is utilized, that is, the meaningful structure in the image is learned faster than random noise (that is, random noise is difficult to be learned), and it is possible to produce a noise-reduced image through unsupervised learning.

[0008] Non-patent document 1 describes a technique for reducing the noise of an object image (PET image) by using the DIP technique. The technique described in the document performs the noise reduction process in two steps. In the first step, for each of a plurality of groups of first input images (MRI images) and teacher images (PET images), the first input image is input to a CNN and a first output image is produced by the CNN, and the CNN is trained (supervised prior learning) based on the evaluation result of the error between the first output image and the teacher image.

[0009] In the next second step, the second input image (MRI image) is input to the CNN that has been previously learned in the first step, and the second output image is produced by the CNN, and the CNN is further learned based on the evaluation result of the error between the second output image and the target image (PET image) (unsupervised learning). Then, the second output image is set as a denoised image that reduces the noise of the target image (PET image).

[0010] In summary, the denoising technique described in non-patent document 1 is to make a denoised image with reduced noise in the target image by making the CNN learn (supervised prior learning) by converting the first input image into a teacher image in the first step, and making the CNN learn further (unsupervised learning) in the second step. Therefore, the denoising technique described in non-patent document 1 can improve the denoising performance compared with the case of only DIP technology (only unsupervised learning).

[0011] Prior art literature

[0012] Non-patent literature

[0013] Non-patent document 1: J.Cui et al., "Populational and individual informationbased PET image denoising using conditional unsupervised learning", Phys.Med.Biol.66 155001, 2021 Summary of the invention

[0014] Problems to be solved by the invention

[0015] The denoising technique described in Non-Patent Document 1 requires not only the preparation of a plurality of teacher images but also the preparation of a plurality of first input images in order to perform the supervised prior learning in the first step. However, it is not easy to prepare a plurality of first input images separately from the preparation of a plurality of teacher images.

[0016] An object of the present invention is to provide an image processing device and an image processing method that can easily reduce the noise of a target image by performing unsupervised learning on a CNN after supervised pre-learning.

[0017] Technical means of solving problems

[0018] The embodiment of the present invention is an image processing device. The image processing device is an image processing device that reduces the noise of a target image to produce a denoised image, and comprises: (1) a first CNN processing unit, which inputs a first input image in which the pixel values ​​of a part of the region are changed based on a teacher image into a convolutional neural network and produces a first output image through the convolutional neural network; (2) a first CNN learning unit, which evaluates the error between the first output image and the teacher image and causes the convolutional neural network to learn based on the error evaluation result; (3) a second CNN processing unit, which inputs a second input image into the convolutional neural network and produces a second output image through the convolutional neural network; and (4) a second CNN learning unit, which evaluates the error between the second output image and the target image and causes the convolutional neural network to learn based on the error evaluation result, and after repeatedly performing the processing of the first CNN processing unit and the first CNN learning unit on each of a plurality of groups of the teacher image and the first input image for a plurality of times, repeatedly performing the processing of the second CNN processing unit and the second CNN learning unit for a plurality of times, and setting the second output image as the denoised image.

[0019] The embodiment of the present invention is an image processing method. The image processing method is an image processing method for reducing the noise of an object image to produce a denoised image, comprising: (1) a first CNN processing step, which inputs a first input image whose pixel values ​​of a part of the region are changed based on a teacher image into a convolutional neural network and produces a first output image through the convolutional neural network; (2) a first CNN learning step, which evaluates the error between the first output image and the teacher image and causes the convolutional neural network to learn based on the error evaluation result; (3) a second CNN processing step, which inputs a second input image into the convolutional neural network and produces a second output image through the convolutional neural network; and (4) a second CNN learning step, which evaluates the error between the second output image and the object image and causes the convolutional neural network to learn based on the error evaluation result, after repeatedly performing the first CNN processing step and the first CNN learning step on each of a plurality of groups of the teacher image and the first input image for a plurality of times, repeatedly performing the second CNN processing step and the second CNN learning step for a plurality of times, and setting the second output image as the denoised image.

[0020] Effects of the Invention

[0021] According to an embodiment of the present invention, an input image to a convolutional neural network (CNN) used in supervised prior learning of the CNN can be easily produced based on a teacher image, and the noise of the object image can be easily reduced by performing unsupervised learning on the CNN after the supervised prior learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a diagram showing the structure of the image processing device 1 .

[0023] Figure 2 is a flow chart of the image processing method.

[0024] Figure 3 This is a diagram showing a structural example of a CNN.

[0025] Figure 4 (a) to (c) are diagrams showing input images (MRI images) to CNN.

[0026] Figure 5 (a) to (c) are diagrams showing phantom images (correct images).

[0027] Figure 6 (a) to (c) represent the simulation of tumors and the administration of drugs 18 Figure 3 is a tomographic image (target image) of the head using F-FDG.

[0028] Figure 7 (a) to (c) are diagrams showing tomographic images after noise reduction processing is performed using the image processing method of Comparative Example 1.

[0029] Figure 8 (a) to (c) are diagrams showing tomographic images after noise reduction processing is performed using the image processing method of Comparative Example 2.

[0030] Fig. 9 (a) to (c) are diagrams showing tomographic images after noise reduction processing has been performed according to the image processing method of the embodiment.

[0031] Fig.10 This is a table summarizing the values ​​of PSNR, SSIM, and CNR of tomographic images after noise reduction processing according to the image processing methods of the embodiment and comparative examples 1 and 2.

[0032] Fig.11 (a) to (c) are diagrams showing input images (MRI images) to CNN.

[0033] Fig.12 (a) to (c) indicate that the drug was administered 18 F~Diagram of a tomographic image (target image) of the head of AV45.

[0034] Fig.13 (a) to (c) are diagrams showing tomographic images after noise reduction processing is performed using the image processing method of Comparative Example 1.

[0035] Fig.14 (a) to (c) are diagrams showing tomographic images after noise reduction processing is performed using the image processing method of Comparative Example 2.

[0036] Fig.15 (a) to (c) are diagrams showing tomographic images after noise reduction processing has been performed according to the image processing method of the embodiment.

[0037] Fig.16 (a) and (b) are diagrams showing input images (MRI images) to CNN.

[0038] Fig.17 (a) and (b) indicate that the drug was administered 11 Figure showing a tomographic image (target image) of the head of C-PIB.

[0039] Fig.18 (a) and (b) are diagrams showing tomographic images after noise reduction processing is performed according to the image processing method of Comparative Example 1.

[0040] Fig.19 (a) and (b) are diagrams showing tomographic images after noise reduction processing is performed according to the image processing method of Comparative Example 2.

[0041] Fig. 20 (a) and (b) are diagrams showing tomographic images after noise reduction processing has been performed according to the image processing method of the embodiment.

[0042] Fig.21 (a) and (b) are diagrams showing input images (MRI images) to CNN.

[0043] Fig. 22 (a) and (b) indicate that the drug was administered 18 Figure 3 is a tomographic image (target image) of the head using F-FDG.

[0044] Fig.23 (a) and (b) are diagrams showing tomographic images after noise reduction processing is performed according to the image processing method of Comparative Example 1.

[0045] Fig.24 (a) and (b) are diagrams showing tomographic images after noise reduction processing is performed according to the image processing method of Comparative Example 2.

[0046] Fig.25(a) and (b) are diagrams showing tomographic images after noise reduction processing has been performed according to the image processing method of the embodiment.

[0047] Fig.26 TABLE 2 is a table summarizing CNR values ​​of tomographic images after noise reduction processing according to the image processing methods of the embodiment and comparative examples 1 and 2. DETAILED DESCRIPTION

[0048] Hereinafter, embodiments of the image processing device and the image processing method will be described in detail with reference to the accompanying drawings. In addition, in the description of the accompanying drawings, the same elements are marked with the same symbols, and repeated descriptions are omitted. The present invention is not limited to these examples, and is intended to include all changes within the meaning and scope of the claims and equivalent to the claims.

[0049] Figure 1 2 is a diagram showing the configuration of the image processing device 1. The image processing device 1 includes an input image creating unit 10, a first computing unit 20, and a second computing unit 30, and reduces noise of a target image 52 to create a noise-reduced image.

[0050] In the figure, a PET image of the head is shown as an example of each of the first input image 40, the first output image 41, the teacher image 42, the second output image 51, and the object image 52, and an MRI image of the head is shown as an example of the second input image 50. The following description is based on this premise. However, the present embodiment is not limited to this.

[0051] This embodiment uses a tomographic image of a subject (eg, a PET image) reconstructed based on information acquired by a radiation tomography apparatus as a target image and is suitable for reducing noise in the target image. These images may be two-dimensional images or three-dimensional images.

[0052] The image processing device 1 includes a GPU (Graphics Processing Unit) for performing processing using CNN, an input unit (such as a keyboard or a mouse) for receiving input from an operator, a display unit (such as a liquid crystal display) for displaying images, etc., and a storage unit for storing programs or data for executing various processes. As the image processing device 1, a computer having a CPU, RAM, ROM, a hard disk drive, etc. can be used.

[0053] The input image creating unit 10 creates the first input image 40 in which the pixel values ​​of a part of the region are changed based on the training image 42 .

[0054] The first operation unit 20 includes a first CNN processing unit 21 and a first CNN learning unit 22, and performs supervised prior learning. The first CNN processing unit 21 inputs the first input image 40 to the CNN, and generates a first output image 41 through the CNN. The first CNN learning unit 22 evaluates the error between the first output image 41 and the teacher image 42, and learns the CNN based on the error evaluation result.

[0055] The first calculation unit 20 repeatedly performs the processing of the first CNN processing unit 21 and the first CNN learning unit 22 a plurality of times on each of a plurality of sets of the training image 42 and the first input image 40 .

[0056] The second operation unit 30 includes a second CNN processing unit 31 and a second CNN learning unit 32, and performs unsupervised learning. The second CNN processing unit 31 inputs the second input image 50 to the CNN, and generates a second output image 51 through the CNN. The second CNN learning unit 32 evaluates the error between the second output image 51 and the target image 52, and learns the CNN based on the error evaluation result.

[0057] After the repetitive processing of the first operation unit 20 is completed, the second operation unit 30 sets the learning state of the CNN at the end point to the initial value, and repeatedly performs the processing of the second CNN processing unit 31 and the second CNN learning unit 32. Then, the second output image 51 at the end point of the repetitive processing of the second operation unit 30 is set as the noise reduction image.

[0058] Figure 2 2 is a flowchart of an image processing method. The image processing method reduces the noise of the target image 52 and creates a noise-reduced image by sequentially performing an input image creation step S10, a first calculation step S20, and a second calculation step S30.

[0059] The input image creation step S10 is a process performed by the input image creation unit 10. In the input image creation step S10, the first input image 40 is created based on the training image 42 in which the pixel values ​​of a part of the region are changed.

[0060] The first operation step S20 is a process of supervised prior learning performed by the first operation unit 20, and includes a first CNN processing step S21 and a first CNN learning step S22. The first CNN processing step S21 is a process performed by the first CNN processing unit 21. The first CNN learning step S22 is a process performed by the first CNN learning unit 22.

[0061] In the first CNN processing step S21, the first input image 40 is input to the CNN, and the first output image 41 is produced by the CNN. In the first CNN learning step S22, the error between the first output image 41 and the teacher image 42 is evaluated, and the CNN is learned based on the error evaluation result. In the first operation step S20, the first CNN processing step S21 and the first CNN learning step S22 are repeatedly performed multiple times on each of the multiple groups of the teacher image 42 and the first input image 40.

[0062] The second operation step S30 is an unsupervised learning process performed by the second operation unit 30, and includes a second CNN processing step S31 and a second CNN learning step S32. The second CNN processing step S31 is a process performed by the second CNN processing unit 31. The second CNN learning step S32 is a process performed by the second CNN learning unit 32.

[0063] In the second CNN processing step S31, the second input image 50 is input to the CNN, and the second output image 51 is produced by the CNN. In the second CNN learning step S32, the error between the second output image 51 and the target image 52 is evaluated, and the CNN is learned based on the error evaluation result. In the second calculation step S30, after the iterative processing in the first calculation step S20 is completed, the learning state of the CNN at the end point is set to the initial value, and the processing of the second CNN processing step S31 and the second CNN learning step S32 is repeated multiple times. Then, the second output image 51 at the end point of the iterative processing in the second calculation step S30 is set as a denoised image.

[0064] Figure 3 This is a diagram showing an example of the structure of a CNN. As shown in the figure, the CNN is preferably a CNN of a U-net structure including an encoder and a decoder. In the figure, the number of pixels of the input image input to the CNN is set to N×N×64, indicating the size of each layer of the CNN.

[0065] In the first CNN learning step S22, the first CNN learning unit 22 learns both the encoder and the decoder of the CNN. In the second CNN learning step S32, the second CNN learning unit 32 can learn both the encoder and the decoder of the CNN, but in order to shorten the time required for learning, it is preferred to selectively learn the decoder of the encoder and the decoder of the CNN.

[0066] In each of the first CNN learning unit 22 in the first CNN learning step S22 and the second CNN learning unit 32 in the second CNN learning step S32, the function for evaluating the error between the two images may be arbitrary. The error evaluation function may use, for example, the L1 norm, the L2 norm, the negative log-likelihood in the Poisson distribution, and the like.

[0067] In the image processing device 1 or the image processing method, each image is as follows. The target image 52 is an image to be subjected to noise reduction, and in this case, is a PET image of the head. Such a PET image is generally composed of a region to be subjected to noise reduction (noise reduction target region), that is, an image region of the head, and a background region around the head region.

[0068] The teacher image 42 is a PET image of the head similar to the target image 52. A plurality of teacher images 42 may be prepared, or a single teacher image 42 may be prepared. The target image 52 itself may be used as the teacher image 42.

[0069] The first input image 40 is an image in which the pixel values ​​of a part of the region are changed based on the teacher image 42, and preferably an image in which the pixel values ​​of a part of the noise reduction target region are changed based on the teacher image 42. A plurality of first input images 40 may be prepared for each teacher image 42.

[0070] The pixel value of the first input image 40 relative to the teacher image 42 may be changed in any manner. The partial region where the pixel value is changed may be of any shape, any size, and any number. The change of the pixel value in the partial region may be, for example, a nonlinear transformation of the pixel value, an exchange of the pixel values ​​between a plurality of pixels, a certain displacement of the pixel value, a random displacement of the pixel value, etc.

[0071] The pixel value changing method (e.g., the shape, size, and number of the partial area where the pixel value is changed, the pixel value changing method) is preferably different between the plurality of first input images 40. In the input image making step S10, the input image making unit 10 makes such a first input image 40 based on the teacher image 42.

[0072] The second input image 50 may be an image representing the morphological information of the subject, such as Figure 1 The MRI image shown in , may also be a CT image or a static PET image. The second input image 50 may also be a random noise image.

[0073] Next, use Figures 4 to 10 , the results of creating a target image by Monte Carlo simulation of a head PET device using a digital brain phantom image, and using the target image to perform noise reduction processing using the image processing methods of Example and Comparative Examples 1 and 2 are described. The phantom image was obtained from BrainWeb (https: / / brainweb.bic.mni.mcgill.ca / brainweb / ).

[0074] The image processing method of the embodiment is based on the above-mentioned embodiment. The image processing method of the comparative example 1 is based on the existing DIP technology. The image processing method of the comparative example 2 is based on the technology described in non-patent document 1.

[0075] In the supervised pre-learning in the image processing methods of the embodiment and the comparative example 2, 20 teacher images produced based on the model images were used. In the image processing method of the embodiment, 32 first input images produced by changing the pixel values ​​in various ways with respect to each teacher image were used. In the image processing method of the comparative example 2, MRI images were used as input images to the CNN.

[0076] In the unsupervised learning in the image processing methods of the embodiment and comparative examples 1 and 2, a simulated image of a patient having a tumor and being administered a drug was used as the target image. 18 F-FDG head cross-sectional image. MRI images were used as input images to CNN.

[0077] Figures 4 to 9 These are diagrams showing tomographic images used or created in unsupervised learning in the image processing methods of the embodiment and comparative examples 1 and 2, respectively. Figure 4 This is a diagram showing an input image (MRI image) to CNN. Figure 5 A diagram showing a phantom image (correct image). Figure 6 It means that the tumor is simulated and the drug is administered 18 Figure 3 is a tomographic image (target image) of the head using F-FDG.

[0078] Figure 7 FIG. 3 is a diagram showing a tomographic image after noise reduction processing is performed using the image processing method of Comparative Example 1. FIG. Figure 8 FIG. 4 is a diagram showing a tomographic image after noise reduction processing is performed using the image processing method of Comparative Example 2. FIG. Fig. 9 FIG. 4 is a diagram showing a tomographic image after noise reduction processing according to the image processing method of the embodiment. Figures 4 to 9 In each of the images, (a) is a cross-sectional image, (b) is a coronal cross-sectional image, and (c) is a sagittal cross-sectional image.

[0079] Fig.10This is a table summarizing the values ​​of PSNR, SSIM and CNR of tomographic images after noise reduction processing according to the image processing methods of the embodiment and comparative examples 1 and 2. PSNR (Peak Signal to Noise Ratio) expresses the quality of the image in decibels (dB). SSIM (structural similarity index) is an indicator that quantifies the brightness and contrast of an image and the changes in its structure. CNR (Contrast to Noise Ratio) is the ratio of the contrast to the noise of an image. For PSNR, SSIM and CNR, the larger the value, the better the image quality.

[0080] like Figures 4 to 10 As shown, it can be confirmed that in the embodiment, compared with Comparative Examples 1 and 2, the PSNR, SSIM, and CNR are all large, and the noise of the target image can be reduced more effectively.

[0081] Next, use Figures 11 to 26 The results of noise reduction processing performed by the image processing methods of the embodiment and comparative examples 1 and 2 using clinical data will be described.

[0082] In the supervised pre-learning in the image processing methods of the embodiment and the comparative example 2, 24 tomographic images of clinical data were used as teacher images. These tomographic images were taken as 18 PET image of the head of F-AV45. In the image processing method of the embodiment, 32 first input images were used, which were created by changing the pixel values ​​in different ways with respect to each teacher image. In the image processing method of comparative example 2, MRI images were used as input images to CNN.

[0083] In the unsupervised learning in the image processing methods of the embodiment and comparative examples 1 and 2, as the target image, a patient who had been administered a drug was used. 18 F-AV45, 11 C-PIB and 18 A tomographic image of the head of any of the F-FDG. MRI images were used as input images to the CNN.

[0084] Figure 11 to Figure 15 They are respectively used as a drug 18 A diagram showing tomographic images used or created in unsupervised learning in the image processing methods of each of Example and Comparative Examples 1 and 2 when a tomographic image of the head of F to AV45 is used as a target image. Fig.11 This is a diagram showing an input image (MRI image) to CNN. Fig.12It means that it is used as a medicine. 18 Figure showing a tomographic image (target image) of the head of F-AV45.

[0085] Fig.13 FIG. 3 is a diagram showing a tomographic image after noise reduction processing is performed using the image processing method of Comparative Example 1. FIG. Fig.14 FIG. 4 is a diagram showing a tomographic image after noise reduction processing is performed using the image processing method of Comparative Example 2. FIG. Fig.15 FIG. 4 is a diagram showing a tomographic image after noise reduction processing according to the image processing method of the embodiment. Figure 11 to Figure 15 In each of the images, (a) is a cross-sectional image, (b) is a coronal cross-sectional image, and (c) is a sagittal cross-sectional image.

[0086] Figures 16 to 20 They are respectively used as a drug 11 A diagram showing tomographic images used or created in unsupervised learning in the image processing methods of each of the embodiment and comparative examples 1 and 2 when a tomographic image of the head of C-PIB is used as a target image. Fig.16 This is a diagram showing an input image (MRI image) to CNN. Fig.17 It means that it is used as a medicine. 11 Figure showing a tomographic image (target image) of the head of C-PIB.

[0087] Fig.18 FIG. 3 is a diagram showing a tomographic image after noise reduction processing is performed using the image processing method of Comparative Example 1. FIG. Fig.19 FIG. 4 is a diagram showing a tomographic image after noise reduction processing is performed using the image processing method of Comparative Example 2. FIG. Fig. 20 FIG. 4 is a diagram showing a tomographic image after noise reduction processing according to the image processing method of the embodiment. Figures 16 to 20 In the figure, (a) is a cross-sectional image, and (b) is a sagittal cross-sectional image.

[0088] Figure 21 to Figure 25 They are respectively used as a drug 18 A diagram showing tomographic images used or created in unsupervised learning in the image processing methods of each of Example and Comparative Examples 1 and 2 when a tomographic image of the head of F-FDG is used as a target image. Fig.21 This is a diagram showing an input image (MRI image) to CNN. Fig. 22 It means that it is used as a medicine. 18 Figure 3 is a tomographic image (target image) of the head using F-FDG.

[0089] Fig.23 FIG. 3 is a diagram showing a tomographic image after noise reduction processing is performed using the image processing method of Comparative Example 1. FIG. Fig.24FIG. 4 is a diagram showing a tomographic image after noise reduction processing is performed using the image processing method of Comparative Example 2. FIG. Fig.25 FIG. 4 is a diagram showing a tomographic image after noise reduction processing according to the image processing method of the embodiment. Figure 21 to Figure 25 In the figure, (a) is a cross-sectional image, and (b) is a sagittal cross-sectional image.

[0090] Fig.26 TABLE 1 is a table summarizing the CNR values ​​of tomographic images after noise reduction processing according to the image processing methods of the embodiment and comparative examples 1 and 2. 18 F-AV45, 11 C-PIB and 18 The CNR values ​​are shown for the cases where F-FDG was administered to the head.

[0091] according to Figures 11 to 26 The following is known. Compared with Comparative Examples 1 and 2, it can be confirmed that the CNR value is large in the embodiment, and the noise of the object image can be reduced more effectively. 18 When the tomographic image when F-FDG is applied to the head is used as the target image, the improvement of CNR in Comparative Example 2 is small compared with Comparative Example 1, whereas the improvement of CNR in the embodiment is large compared with Comparative Examples 1 and 2.

[0092] In this way, when the types of medicines applied to the head between the teacher image and the object image are different, it can be confirmed that in Comparison Example 2, the degree of noise removal is the same as that in Comparison Example 1. In contrast, in the embodiment, the degree of noise removal is improved compared with Comparison Examples 1 and 2.

[0093] As described above, according to the present embodiment, an input image to a CNN used in supervised pre-learning of the CNN can be easily prepared based on the teacher image, so that the noise of the object image can be easily reduced by performing unsupervised learning on the CNN after supervised pre-learning. In addition, even when the types of drugs administered to the subject are different between the teacher image and the object image, the noise of the object image can be effectively reduced.

[0094] The image processing device and the image processing method are not limited to the above-described embodiments and configuration examples, and various modifications are possible.

[0095] The image processing device according to the first mode of the above-mentioned embodiment is an image processing device for reducing the noise of an object image to produce a denoised image, and comprises: (1) a first CNN processing unit, which inputs a first input image whose pixel values ​​of a part of the area are changed based on a teacher image into a convolutional neural network and produces a first output image through the convolutional neural network; (2) a first CNN learning unit, which evaluates the error between the first output image and the teacher image and causes the convolutional neural network to learn based on the error evaluation result; (3) a second CNN processing unit, which inputs a second input image into the convolutional neural network and produces a second output image through the convolutional neural network; and (4) a second CNN learning unit, which evaluates the error between the second output image and the object image and causes the convolutional neural network to learn based on the error evaluation result, after repeatedly performing the processing of each of the first CNN processing unit and the first CNN learning unit multiple times on each of the multiple groups of the teacher image and the first input image, and then repeatedly performing the processing of each of the second CNN processing unit and the second CNN learning unit multiple times, and setting the second output image as the denoised image.

[0096] The image processing device of the second aspect may be configured so that, in the configuration of the first aspect, further comprises an input image creating unit that creates a first input image in which pixel values ​​of a part of a region are changed based on the teacher image.

[0097] In the image processing device of the third embodiment, it can also be configured that, in the structure of the first or second embodiment, the convolutional neural network is a convolutional neural network of a U-net structure including an encoder and a decoder, the first CNN learning unit learns both the encoder and the decoder of the convolutional neural network, and the second CNN learning unit selectively learns the decoder of the convolutional neural network.

[0098] In the image processing device of the fourth mode, it can also be constructed that, in any structure of the first to third modes, the first CNN processing unit inputs a plurality of first input images whose pixel value changing methods are different from each other into the convolutional neural network for each teacher image, and produces a first output image for each first input image through the convolutional neural network.

[0099] In the image processing device of the fifth mode, it can also be configured that, in any structure of the first to fourth modes, the first CNN processing unit inputs a first input image in which pixel values ​​of a part of the denoising target area are changed based on a teacher image into the convolutional neural network.

[0100] In the image processing device of the sixth aspect, in any one of the configurations of the first to fifth aspects, the target image and the teacher image may be tomographic images of the subject reconstructed based on information acquired by a radiation tomography apparatus.

[0101] The image processing method of the first mode according to the above-mentioned embodiment is an image processing method for reducing the noise of the object image to produce a denoised image, comprising: (1) a first CNN processing step, which inputs a first input image whose pixel values ​​of a part of the area are changed based on a teacher image into a convolutional neural network and produces a first output image through the convolutional neural network; (2) a first CNN learning step, which evaluates the error between the first output image and the teacher image and causes the convolutional neural network to learn based on the error evaluation result; (3) a second CNN processing step, which inputs a second input image into the convolutional neural network and produces a second output image through the convolutional neural network; and (4) a second CNN learning step, which evaluates the error between the second output image and the object image and causes the convolutional neural network to learn based on the error evaluation result, after repeatedly performing the first CNN processing step and the first CNN learning step on each of the multiple groups of the teacher image and the first input image for multiple times, and then repeatedly performing the second CNN processing step and the second CNN learning step for multiple times, so that the second output image is set as the denoised image.

[0102] The image processing method of the second aspect may be configured such that, in the configuration of the first aspect, further comprises: an input image creating step of creating a first input image in which pixel values ​​of a part of a region are changed based on the teacher image.

[0103] In the image processing method of the third embodiment, it can also be constructed that in the structure of the first or second embodiment, the convolutional neural network is a convolutional neural network of a U-net structure including an encoder and a decoder, in the first CNN learning step, both the encoder and the decoder of the convolutional neural network are learned, and in the second CNN learning step, the decoder of the convolutional neural network is selectively learned.

[0104] In the image processing method of the fourth mode, it can also be constructed that in any structure of the first to third modes, in the first CNN processing step, for each teacher image, each of a plurality of first input images in which the manner of changing pixel values ​​is different from each other is input into the convolutional neural network, and a first output image is produced by the convolutional neural network for each first input image.

[0105] In the image processing method of the fifth mode, it can also be constructed that in any structure of the first to fourth modes, in the first CNN processing step, a first input image in which the pixel values ​​of a part of the denoising object area are changed based on the teacher image is input into the convolutional neural network.

[0106] In the image processing method of the sixth aspect, in any one of the configurations of the first to fifth aspects, the target image and the teacher image may be tomographic images of the subject reconstructed based on information acquired by a radiation tomography apparatus.

[0107] Industrial Applicability

[0108] The present invention can be used as an image processing device and an image processing method that can easily reduce the noise of a target image by performing unsupervised learning on a CNN after supervised pre-learning.

[0109] Explanation of symbols

[0110] 1 ... image processing device, 10 ... input image generating unit, 20 ... first computing unit, 21 ... first CNN processing unit, 22 ... first CNN learning unit, 30 ... second computing unit, 31 ... second CNN processing unit, 32 ... second CNN learning unit.

Claims

1. An image processing device, wherein: An image processing device that reduces the noise of an object image to produce a noise-reduced image. have: A first CNN processing unit, which inputs a first input image in which pixel values ​​of a part of an area are changed based on a teacher image into a convolutional neural network and generates a first output image through the convolutional neural network; a first CNN learning unit that evaluates an error between the first output image and the teacher image and causes the convolutional neural network to learn based on the error evaluation result; a second CNN processing unit, which inputs a second input image into the convolutional neural network and generates a second output image through the convolutional neural network; as well as a second CNN learning unit that evaluates an error between the second output image and the target image and causes the convolutional neural network to learn based on the error evaluation result; After repeatedly performing the processing by the first CNN processing unit and the first CNN learning unit on each of the multiple groups of the teacher image and the first input image, repeatedly performing the processing by the second CNN processing unit and the second CNN learning unit on each of the multiple groups, the second output image is set to the denoised image.

2. The image processing device according to claim 1, wherein: The method further includes an input image generating unit configured to generate a first input image in which pixel values ​​of a part of a region are changed based on the training image.

3. The image processing device according to claim 1 or 2, wherein: The convolutional neural network is a convolutional neural network with a U-net structure including an encoder and a decoder. The first CNN learning unit causes both the encoder and the decoder of the convolutional neural network to learn. The second CNN learning unit selectively learns the decoder of the convolutional neural network.

4. The image processing device according to any one of claims 1 to 3, wherein: The first CNN processing unit inputs a plurality of first input images having different pixel value change modes for each teacher image into the convolutional neural network, and generates a first output image for each first input image through the convolutional neural network.

5. The image processing device according to any one of claims 1 to 4, wherein: The first CNN processing unit inputs a first input image in which pixel values ​​of a part of a noise reduction target area are changed based on a teacher image into the convolutional neural network.

6. The image processing device according to any one of claims 1 to 5, wherein: The target image and the teacher image are tomographic images of the subject reconstructed based on information acquired by a radiation tomography apparatus.

7. An image processing method, wherein: It is an image processing method that reduces the noise of the target image to produce a denoised image. have: A first CNN processing step, which inputs a first input image whose pixel values ​​of a part of the area are changed based on the teacher image into a convolutional neural network and generates a first output image through the convolutional neural network; a first CNN learning step of evaluating an error between the first output image and the teacher image and causing the convolutional neural network to learn based on the error evaluation result; A second CNN processing step, which inputs a second input image into the convolutional neural network and produces a second output image through the convolutional neural network; as well as a second CNN learning step of evaluating an error between the second output image and the object image and causing the convolutional neural network to learn based on the error evaluation result, After repeatedly performing the first CNN processing step and the first CNN learning step on each of the multiple groups of the teacher image and the first input image, repeatedly performing the second CNN processing step and the second CNN learning step on each of the multiple groups, the second output image is set as the denoised image.

8. The image processing method according to claim 7, wherein: The method further comprises: an input image creating step of creating a first input image in which pixel values ​​of a part of a region are changed based on the teacher image.

9. The image processing method according to claim 7 or 8, wherein: The convolutional neural network is a convolutional neural network with a U-net structure including an encoder and a decoder. In the first CNN learning step, both the encoder and the decoder of the convolutional neural network are learned, In the second CNN learning step, the decoder of the convolutional neural network is selectively learned.

10. The image processing method according to any one of claims 7 to 9, wherein: In the first CNN processing step, for each teacher image, a plurality of first input images having pixel values ​​changed in different ways are respectively input into the convolutional neural network, and a first output image is produced by the convolutional neural network for each first input image.

11. The image processing method according to any one of claims 7 to 10, wherein: In the first CNN processing step, a first input image in which pixel values ​​of a part of a denoising target area are changed based on a teacher image is input to the convolutional neural network.

12. The image processing method according to any one of claims 7 to 11, wherein: The target image and the teacher image are tomographic images of the subject reconstructed based on information acquired by a radiation tomography apparatus.

Images