Image processing device, image processing method, and tomographic image acquisition system

Abstract

To provide an image processing device which can acquire a tomographic image with excellent S / N in a short time by using a TOF-PET device.SOLUTION: A tomographic image acquisition system 1 comprises an image processing device 10, a TOF-PET device 20, a CT device 30 and an MRI device 40. The image processing device 10 estimates a tomographic image of a test subject with a deep neural network on the basis of an input image group and outputs the estimated tomographic image. The input image group at least includes a plurality of γ ray pair generation position distribution image of the test subject acquired correspondingly to each of a plurality of regions sectioned about the azimuth angle θ of the Line of Response in the TOF-PET device 20. The input image group may include a γ ray absorption coefficient distribution image of the test subject and may include an MRI image of the test subject.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing method, and a tomographic image acquisition system. [Background technology]

[0002] A PET (Positron Emission Tomography) device is equipped with a number of radiation detectors surrounding a measurement space. A subject who has been administered a drug labeled with a positron-emitting nuclide is placed in the measurement space. Two gamma-ray photons with an energy of 511 KeV are generated by electron-positron annihilation in the subject's body, and these two gamma-ray photons fly in opposite directions. The PET device counts these two gamma-ray photons by coincidence counting using any two of the radiation detectors. The PET device collects such coincidence information and performs the required image reconstruction processing to obtain a tomographic image of the subject.

[0003] Among PET devices, a TOF-PET (Time-of-Flight PET) device can detect the gamma-ray pair generation position on the coincidence line connecting two radiation detectors that have counted a gamma-ray pair for each coincidence event, based on the time difference between the detection timings of the two radiation detectors that have counted a gamma-ray pair simultaneously. The TOF-PET device can obtain a gamma-ray pair generation position distribution image that represents the distribution of gamma-ray pair generation positions by detecting the gamma-ray pair generation positions for a large number of coincidence events. The gamma-ray pair generation position distribution image becomes a blurred image according to the TOF resolution. This blurring can be removed by image reconstruction processing to obtain a tomographic image for clinical use. Non-Patent Documents 1 and 2 describe a technology for obtaining a tomographic image of a subject in a short time using a TOF-PET device. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] William Whiteley, et al,"FastPET: Near Real-Time Reconstruction of PET Histo-Image Data Using aNeural Network," IEEE TRANSACTIONS ON RADIATION& PLASMA MEDICAL SCIENCES, October 2020. [Non-Patent Document 2] Eiichi Tanaka, "Line-WritingData Acquisition and Signal-to-Noise Ratio in Time-of-Flight Positron Emission Tomography," Workshop on Time-of-Flight Tomography, May 1982. [Non-Patent Document 3] "BrainWeb: Simulated Brain Database", [online], [searched on March 15, 2021], Internet<https: / / brainweb.bic.mni.mcgill.ca / brainweb / > Summary of the Invention [Problem to be solved by the invention]

[0005] In the conventional technique of acquiring tomographic images of a subject in a short time using a TOF-PET device, the signal-to-noise ratio of the final tomographic image is poor.

[0006] The present invention has been made to solve the above problems, and aims to provide an image processing device and an image processing method that can obtain tomographic images with a good S / N ratio in a short time using a TOF-PET device, and also aims to provide a tomographic image acquisition system equipped with such an image processing device and TOF-PET device. [Means for solving the problem]

[0007] The image processing device of the present invention includes an input unit that inputs a group of input images including a plurality of gamma-ray pair generation position distribution images of a subject obtained corresponding to each of a plurality of regions divided in a TOF-PET device in relation to the azimuth angle of coincidence lines, an estimation unit that estimates a tomographic image of the subject using a deep neural network based on the group of input images input to the input unit, and an output unit that outputs the tomographic image estimated by the estimation unit.

[0008] The input image group input to the input unit may further include a gamma ray absorption coefficient distribution image of the subject, or may further include an MRI image of the subject.

[0009] It is preferable that the estimation unit weights a feature map generated during processing by a first deep neural network that estimates a tomographic image based on a plurality of gamma-ray pair generation location distribution images with a feature map generated during processing by a second deep neural network that inputs an MRI image, thereby generating a new feature map.

[0010] It is preferable that the estimation unit performs weighting in each of a plurality of layers in the first deep neural network, each of which has a feature map with a different resolution.

[0011] It is preferable that the estimation unit weights each of a plurality of layers having feature maps with different resolutions in the first deep neural network using the last feature map of each of a plurality of layers having feature maps with different resolutions in the second deep neural network, and sets a new feature map created by the weighting as the last feature map of that layer.

[0012] It is preferable that the image processing device of the present invention further includes a learning unit that trains the deep neural network using a database of input images and tomographic images acquired for each of a plurality of subjects.

[0013] The tomographic image acquisition system of the present invention includes a TOF-PET device that acquires a plurality of gamma-ray pair generation position distribution images of a subject corresponding to each of a plurality of regions divided in relation to the azimuth angle of the coincidence lines, and the image processing device of the present invention described above that infers a tomographic image of the subject based on the plurality of gamma-ray pair generation position distribution images.

[0014] The tomographic image acquisition system of the present invention may be configured to include a TOF-PET device that acquires a plurality of gamma-ray pair generation position distribution images of a subject corresponding to each of a plurality of regions divided in relation to the azimuth angle of the coincidence lines, a device that acquires a gamma-ray absorption coefficient distribution image of the subject, and the above-mentioned image processing device of the present invention that infers a tomographic image of the subject based on the plurality of gamma-ray pair generation position distribution images and the gamma-ray absorption coefficient distribution image.

[0015] The tomographic image acquisition system of the present invention may be configured to include a TOF-PET device that acquires a plurality of gamma-ray pair generation position distribution images of a subject corresponding to each of a plurality of regions divided in relation to the azimuth angle of the coincidence lines, an MRI device that acquires an MRI image of the subject, and the above-mentioned image processing device of the present invention that infers a tomographic image of the subject based on the plurality of gamma-ray pair generation position distribution images and the MRI image.

[0016] The image processing method of the present invention includes an input step of inputting a group of input images including a plurality of gamma-ray pair generation position distribution images of a subject obtained corresponding to each of a plurality of regions divided in a TOF-PET device in relation to the azimuth angle of coincidence lines, an inference step of inferring a tomographic image of the subject using a deep neural network based on the group of input images input in the input step, and an output step of outputting the tomographic image inferred in the inference step.

[0017] The input image group input in the input step may further include a gamma ray absorption coefficient distribution image of the subject, or may further include an MRI image of the subject.

[0018] In the estimation step, it is preferable that a feature map created during processing by a first deep neural network that infers a tomographic image based on a plurality of gamma-ray pair generation location distribution images is weighted by a feature map created during processing by a second deep neural network to which an MRI image is input, thereby creating a new feature map.

[0019] The estimation step preferably performs weighting in each of a plurality of layers in the first deep neural network having feature maps with different resolutions.

[0020] The estimation step preferably performs weighting for each of a plurality of layers in the first deep neural network having feature maps with different resolutions using the final feature map for each of a plurality of layers in the second deep neural network having feature maps with different resolutions, and sets the new feature map created by the weighting as the final feature map for that layer.

[0021] It is preferable that the image processing method of the present invention further comprises a learning step of training the deep neural network using a database of input images and tomographic images acquired for each of a plurality of subjects. Effect of the Invention

[0022] According to the present invention, a tomographic image with a good S / N ratio can be obtained in a short time using a TOF-PET device. [Brief description of the drawings]

[0023] [Figure 1] FIG. 1 is a diagram showing the configuration of a tomographic image acquisition system 1. As shown in FIG. [Diagram 2] FIG. 2 is a diagram for explaining detection of the position of generation of gamma ray pairs by the TOF-PET device 20. As shown in FIG. [Diagram 3] FIG. 3 is a diagram for explaining acquisition of a distribution image of gamma-ray pair generation positions in each region at an azimuth angle θ by the TOF-PET device 20. In FIG. [Figure 4]FIG. 4 is a diagram showing the configuration of the image processing device 10. As shown in FIG. [Diagram 5] FIG. 5 is a diagram illustrating an example of the configuration of the DNN of the estimation unit 12. [Figure 6] FIG. 6 is a diagram showing another example of the configuration of the DNN of the estimation unit 12. [Figure 7] FIG. 7 is a detailed diagram of a portion of the DNN configuration shown in FIG. [Figure 8] FIG. 8 is a diagram showing eight gamma-ray pair generation position distribution images acquired corresponding to eight regions divided with respect to the azimuth angle θ of coincidence lines in a TOF-PET device. [Figure 9] FIG. 9 is a diagram showing a true radiation source distribution image. [Figure 10] FIG. 10 is a diagram showing a gamma ray absorption coefficient distribution image. [Figure 11] FIG. 11 shows an MRI image. [Figure 12] FIG. 12 is a diagram showing a tomographic image of Comparative Example 1. As shown in FIG. [Figure 13] FIG. 13 is a diagram showing a tomographic image of Comparative Example 2. As shown in FIG. [Figure 14] FIG. 14 is a diagram showing a tomographic image of Comparative Example 3. As shown in FIG. [Figure 15] FIG. 15 is a diagram showing a tomographic image of the first embodiment. [Figure 16] FIG. 16 is a diagram showing a tomographic image of the second embodiment. [Figure 17] FIG. 17 is a graph comparing the average values ​​and standard deviations of the peak signal-to-noise ratios of the tomographic images of Comparative Examples 1 to 3 and Examples 1 and 2. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant descriptions are omitted. The present invention is not limited to these examples, but is shown by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.

[0025] FIG. 1 is a diagram showing the configuration of a tomographic image acquisition system 1. The tomographic image acquisition system 1 includes an image processing apparatus 10 and a TOF-PET apparatus 20. As shown in FIG. 2, the TOF-PET apparatus 20 detects the γ-ray pair generation position on the coincidence line connecting these two radiation detectors based on the time difference between the detection timings of each of the two radiation detectors that have simultaneously counted the γ-ray pair for each coincidence event.

[0026] The TOF-PET apparatus 20 collects the γ-ray pair generation positions for each coincidence event in each of a plurality of regions divided with respect to the azimuth angle of the coincidence line, and acquires a γ-ray pair generation position distribution image of the subject. For example, as shown in FIG. 3, when dividing into 8 regions with respect to the azimuth angle θ of the coincidence line, the TOF-PET apparatus 20 acquires a γ-ray pair generation position distribution image of the subject in each of the regions of 0.0° ≦ θ < 22.5°, 22.5° ≦ θ < 45.0°, 45.0° ≦ θ < 67.5°, 67.5° ≦ θ < 90.0°, 90.0° ≦ θ < 112.5°, 112.5° ≦ θ < 135.0°, 135.0° ≦ θ < 157.5° and 157.5° ≦ θ < 180.0°.

[0027] The image processing device 10 infers a tomographic image of a subject by a deep neural network (DNN) based on an input image group, and outputs the inferred tomographic image. The input image group includes at least a plurality of γ-ray pair generation position distribution images of the subject acquired corresponding to each of a plurality of regions divided with respect to the azimuth angle θ of the coincidence lines in the TOF-PET device 20. Further, the input image group may include a γ-ray absorption coefficient distribution image of the subject, or may include an MRI (Magnetic Resonance Imaging) image of the subject.

[0028] The γ-ray absorption coefficient distribution image of the subject may be acquired by a transmission scan by the TOF-PET device 20, may be acquired based on an X-ray absorption coefficient distribution image of the subject obtained by a CT (Computed Tomography) device, or may also be acquired based on an image representing the structure of the subject obtained by an MRI device.

[0029] The tomographic image acquisition system 1 preferably includes a CT device 30 or an MRI device 40. The CT device 30 is for acquiring an X-ray absorption coefficient distribution image of the subject and acquiring a γ-ray absorption coefficient distribution image of 511 Kev of energy based on this. The MRI device 40 is for acquiring an MRI image representing the structure of the subject, and can also acquire a γ-ray absorption coefficient distribution image based on this MRI image. When acquiring the γ-ray absorption coefficient distribution image of the subject using the TOF-PET device 20 or the MRI device 40, the CT device 30 is unnecessary. Any two or more of the TOF-PET device 20, the CT device 30, and the MRI device 40 may be integrated.

[0030] FIG. 4 is a diagram showing the configuration of the image processing device 10. The image processing device 10 infers a tomographic image of the subject by DNN based on an input image group. Further, the image processing device 10 can train the DNN using the image database 15. The image processing device 10 includes an input unit 11, an inference unit 12, an output unit 13, and a learning unit 14.

[0031] The input unit 11 inputs an input image group including at least a plurality of gamma-ray pair generation position distribution images of the subject for each region divided with respect to the azimuth angle θ of coincidence lines. The input unit 11 may input an input image group including both or either of a gamma-ray absorption coefficient distribution image and an MRI image in addition to the plurality of gamma-ray pair generation position distribution images of the subject.

[0032] The estimation unit 12 estimates a tomographic image of the subject by a DNN based on the input image group input to the input unit 11. This DNN is preferably a Convolutional Neural Network (CNN). In the CNN, convolution layers for extracting features and pooling layers for compressing the features are alternately provided. The estimation unit 12 may process the DNN by a CPU (Central Processing Unit), but it is preferable to perform the process by a DSP (Digital Signal Processor) or GPU (Graphics Processing Unit) that is capable of faster processing.

[0033] The output unit 13 outputs the tomographic image estimated by the estimation unit 12. The output unit 13 preferably includes a display for displaying the image.

[0034] The learning unit 14 trains the DNN of the estimation unit 12 using the image database 15. The image database 15 stores data of input image groups and data of tomographic images for multiple subjects. The learning unit 14 inputs the input image groups of each subject stored in the image database 15 to the input unit 11, and trains the DNN of the estimation unit 12 based on the tomographic images output from the output unit 13 for the input image groups and the tomographic images of the subject stored in the image database. Such learning of the DNN is called deep learning.

[0035] An image processing method using such an image processing device 10 includes an input step by the input unit 11, an estimation step by the estimation unit 12, an output step by the output unit 13, and a learning step by the learning unit 14. That is, in the input step, a plurality of distribution images of gamma-ray pair generation positions of a subject for each region divided with respect to the azimuth angle θ of the coincidence lines are input. In the estimation step, a tomographic image of the subject is estimated by the DNN based on the input image group input in the input step. In the output step, the tomographic image of the subject estimated in the estimation step is output. In the learning step, the DNN is trained using an image database 15.

[0036] Once the DNN has been trained in the learning step, the series of steps of input, inference, and output can be repeated thereafter, so there is no need to perform the learning step each time the series of steps of input, inference, and output are performed. For the same reason, if the DNN has already been trained, the learning unit 14 is not necessary. However, even if the DNN has already been trained, if further training is to be performed to enable more accurate inference, the learning step and the learning unit 14 may be present.

[0037] FIG. 5 is a diagram showing an example of the configuration of the DNN of the estimation unit 12. In the configuration example shown in FIG. 5, an encoder-decoder CNN having a U-net structure is used. FIG. 6 is a diagram showing another example of the configuration of the DNN of the estimation unit 12. In the configuration example shown in FIG. 6, an encoder-decoder CNN (CNN1, CNN2) having two U-net structures is used. FIG. 7 is a diagram showing in detail a part of the DNN configuration shown in FIG. 6. In these figures, feature maps are indicated by boxes, and operations are indicated by arrows.

[0038] In the configuration examples shown in Figures 6 and 7, the first deep neural network (CNN1) inputs a plurality of gamma-ray pair generation position distribution images (and gamma-ray absorption coefficient distribution images) of the subject from the input image group, and infers a tomographic image of the subject. The second deep neural network (CNN2) inputs an MRI image of the subject from the input image group. The estimation unit 12 weights (multiplies each element) the feature map created in the process of processing in CNN1 by the feature map (Attention map) created in the process of processing by CNN2, to create a new feature map.

[0039] It is preferable that the estimation unit 12 performs weighting for each of a plurality of layers having different resolutions of feature maps in CNN1. It is also preferable that the estimation unit 12 performs weighting for each of a plurality of layers having different resolutions of feature maps in CNN1 using the last feature map (Attention map) for each of a plurality of layers having different resolutions of feature maps in CNN2, and sets a new feature map created by the weighting as the last feature map for the layer. It is also preferable that the feature map immediately before weighting in CNN1 is standardized so that the average value of all elements is set to 0 and the standard deviation is set to 1, and it is preferable that the Attention map in CNN2 has each element set to a value between 0 and 1 by a sigmoid function.

[0040] Next, an example will be described. In the example described below, human brain image data (20 cases) published in Non-Patent Document 3 was used, and eight γ-ray pair generation position distribution images, γ-ray absorption coefficient distribution images, and MRI images obtained corresponding to eight regions divided with respect to the azimuth angle θ of coincidence lines in a TOF-PET device were created, and a Monte Carlo simulation was performed.

[0041] The line source distribution was set as gray matter:white matter:cerebrospinal fluid = 1:0.25:0.05. The arrangement of the radiation detectors in the TOF-PET device 20 was the same as that of the head PET device (SHR29000) manufactured by Hamamatsu Photonics K.K. The TOF time resolution in the TOF-PET device 20 was set to 300 ps. The number of pixels in each image was 70×120×120 voxels. The pixel size was 3.221×3.0×3.0 mm 3 was set as such.

[0042] Among the brain image data (20 cases), 13 cases were used for DNN learning, 2 cases were used for learning verification, and 5 cases were used for testing. For data augmentation during learning, random cropping (64×64×64 voxels) was used, and the DNN was trained with a batch size of 32 or 16 using the Adam optimizer. With "2048 / batch size" defined as one epoch, the number of epochs was set to 500. Mean Squared Error (MSE) was used as the loss function representing the difference between the output image of the DNN and the teacher image during training.

[0043] Figure 8 is a diagram showing eight gamma-ray pair generation position distribution images obtained corresponding to eight regions each divided with respect to the azimuth angle θ of the coincidence counting line in the TOF-PET device. As shown in this figure, each gamma-ray pair generation position distribution image is blurred in the direction parallel to the azimuth angle due to the low TOF time resolution, and has a spatial resolution (several millimeters) corresponding to the size of the radiation detector in the direction orthogonal to the azimuth angle.

[0044] Figure 9 is a diagram showing the true line source distribution image. Figure 10 is a diagram showing the gamma-ray absorption coefficient distribution image. Figure 11 is a diagram showing the MRI image.

[0045] Figure 12 is a diagram showing the tomographic image of Comparative Example 1. In Comparative Example 1, a tomographic image of the subject was created without dividing with respect to the azimuth angle θ of the coincidence counting line in the TOF-PET device. Due to the low TOF time resolution, the spatial resolution of the tomographic image in Comparative Example 1 was as low as 4.5 cm.

[0046] 13 is a diagram showing a tomographic image of Comparative Example 2. In Comparative Example 2, a tomographic image of the subject was created by performing image reconstruction processing using the List-Mode Dynamic Row-Action Maximum-Likelihood Algorithm (LM-DRAMA).

[0047] 14 is a diagram showing a tomographic image of Comparative Example 3. In Comparative Example 3, a tomographic image of the subject was created by the method described in Non-Patent Document 1.

[0048] Fig. 15 is a diagram showing a tomographic image in Example 1. In Example 1, the DNN configuration shown in Fig. 5 was used to estimate a tomographic image of a subject. Eight gamma-ray pair generation position distribution images (Fig. 8) and a gamma-ray absorption coefficient distribution image (Fig. 10) were used as an input image group.

[0049] Fig. 16 is a diagram showing a tomographic image of Example 2. In Example 2, the DNN configurations shown in Fig. 6 and Fig. 7 were used to estimate a tomographic image of a subject. As an input image group, eight gamma-ray pair generation position distribution images (Fig. 8), a gamma-ray absorption coefficient distribution image (Fig. 10), and an MRI image (Fig. 11) were used.

[0050] The tomographic images of Examples 1 and 2 (FIGS. 15 and 16) are closer to the true radiation source distribution image (FIG. 9) than the tomographic images of Comparative Examples 1 to 3 (FIGS. 12 to 14).

[0051] FIG. 17 is a graph comparing the average value and standard deviation of the peak signal to noise ratio of the tomographic images of Comparative Examples 1 to 3 and Examples 1 and 2. The peak signal to noise ratio (PSNR) is an image quality expressed in decibels (dB), and the higher the value, the better the image quality. The PSNR of Comparative Example 1 was 21.38 (±0.52) dB, the PSNR of Comparative Example 2 was 27.83 (±0.41) dB, and the PSNR of Comparative Example 3 was 28.82 (±0.24) dB. In contrast, the PSNR of Example 1 was 29.90 (±0.08) dB, and the PSNR of Example 2 was 37.09 (±0.73) dB. The PSNR of the tomographic images of Examples 1 and 2 is higher than the PSNR of the tomographic images of Comparative Examples 1 to 3.

[0052] As described above, in this embodiment, a tomographic image with a good S / N ratio can be obtained using a TOF-PET device. Also, by performing image reconstruction processing using a deep neural network, a tomographic image can be obtained in a short time. [Explanation of symbols]

[0053] 1... tomographic image acquisition system, 10... image processing device, 11... input section, 12... estimation section, 13... output section, 14... learning section, 15... image database, 20... TOF-PET device, 30... CT device, 40... MRI device.

Claims

1. an input unit for inputting an input image group including a plurality of gamma-ray pair generation position distribution images of a subject acquired corresponding to each of a plurality of regions divided in relation to the azimuth angle of coincidence lines in a TOF-PET device; an estimation unit that estimates a tomographic image of the subject using a deep neural network based on the group of input images input to the input unit; an output unit that outputs the tomographic image estimated by the estimation unit; An image processing device comprising:

2. The input image group includes a gamma ray absorption coefficient distribution image of the subject. The image processing device according to claim 1 .

3. the input images include MRI images of the subject; 3. The image processing device according to claim 1 or 2.

4. the estimation unit weights a feature map generated in a process of a first deep neural network estimating the tomographic image based on the plurality of gamma-ray pair generation position distribution images by a feature map generated in a process of a second deep neural network to which the MRI image is input, thereby generating a new feature map; The image processing device according to claim 3 .

5. The estimation unit performs the weighting in each of a plurality of layers having feature maps with different resolutions in the first deep neural network. The image processing device according to claim 4.

6. the estimation unit performs the weighting using a final feature map of each of a plurality of layers having feature maps with different resolutions in the second deep neural network in the first deep neural network, and sets a new feature map created by the weighting as a final feature map of the layer. The image processing device according to claim 5 .

7. The image processing device according to any one of claims 1 to 6, further comprising a learning unit that trains the deep neural network using a database of the input images and tomographic images acquired for each of a plurality of subjects.

8. a TOF-PET device for acquiring a plurality of gamma-ray pair generation position distribution images of a subject corresponding to a plurality of regions divided with respect to the azimuth angle of coincidence lines; an image processing device according to claim 1, which estimates a tomographic image of the subject based on the plurality of gamma ray pair generation position distribution images; A tomographic image acquisition system comprising:

9. a TOF-PET device for acquiring a plurality of gamma-ray pair generation position distribution images of a subject corresponding to a plurality of regions divided with respect to the azimuth angle of coincidence lines; A device for acquiring a gamma ray absorption coefficient distribution image of the subject; an image processing device according to claim 2, which estimates a tomographic image of the subject based on the plurality of gamma-ray pair generation position distribution images and the gamma-ray absorption coefficient distribution image; A tomographic image acquisition system comprising:

10. a TOF-PET device for acquiring a plurality of gamma-ray pair generation position distribution images of a subject corresponding to a plurality of regions divided with respect to the azimuth angle of coincidence lines; an MRI device for acquiring an MRI image of the subject; The image processing device according to any one of claims 3 to 6, which estimates a tomographic image of the subject based on the plurality of gamma-ray pair generation position distribution images and the MRI image; A tomographic image acquisition system comprising:

11. an input step of inputting an input image group including a plurality of gamma-ray pair generation position distribution images of a subject acquired corresponding to each of a plurality of regions divided in relation to the azimuth angle of coincidence lines in a TOF-PET device; an inference step of inferring a tomographic image of the subject by a deep neural network based on the input images input in the input step; an output step of outputting the tomographic image estimated in the estimation step; An image processing method comprising:

12. The input image group includes a gamma ray absorption coefficient distribution image of the subject. The image processing method according to claim 11.

13. the input images include MRI images of the subject; 13. The image processing method according to claim 11 or 12.

14. The estimation step includes weighting a feature map generated during processing by a first deep neural network that estimates the tomographic image based on the plurality of gamma-ray pair generation position distribution images, using a feature map generated during processing by a second deep neural network that receives the MRI image, to generate a new feature map. The image processing method according to claim 13.

15. The estimation step performs the weighting in each of a plurality of layers having different feature map resolutions in the first deep neural network. The image processing method according to claim 14.

16. the estimation step performs the weighting in each of a plurality of layers in the first deep neural network having feature maps with different resolutions by using a final feature map in each of a plurality of layers in the second deep neural network having feature maps with different resolutions, and sets a new feature map created by the weighting as a final feature map in the layer; The image processing method according to claim 15.

17. The image processing method according to any one of claims 11 to 16, further comprising a learning step of training the deep neural network using a database of the input images and tomographic images acquired for each of a plurality of subjects.

Images