Image processing device and image processing method

Abstract

The image processing device (3) of the present invention comprises: a feature extraction unit (12), a reconstruction unit (13), an evaluation unit (14), and a control unit (15). The feature extraction unit (12) inputs an input image z into a feature extraction NN (18), and outputs an intermediate image from the feature extraction NN (18). The reconstruction unit (13) inputs the intermediate image into an m-th reconstruction NN (19 m ), and reconstructed from the mth NN(19 m ) outputs the mth output image (y m,n The evaluation unit (14) evaluates the image based on the m-th tomographic image (x m ) and the mth output image (y m,n The control unit (15) controls the processing of the feature extraction unit (12) and the reconstruction unit (13), the calculation of the evaluation value by the evaluation unit (14), and the calculation of the feature extraction NN (18) and the m-th reconstruction NN (19) based on the evaluation value. m ) is repeatedly learned. Thus, a device is realized that can generate a tomographic image with high-performance noise removal and can shorten the time required for noise removal processing.

Description

Technical Field

[0001] The present disclosure relates to an apparatus and method for generating a tomographic image after noise removal processing based on list data collected by a radiation tomography apparatus. Background Art

[0002] Examples of radiation tomography apparatuses capable of acquiring tomographic images of a subject (living body) include PET (Positron Emission Tomography) apparatuses and SPECT (Single Photon Emission Computed Tomography) apparatuses.

[0003] The PET apparatus includes a detector unit comprising multiple small radiation detectors arranged around a measurement space where a subject is placed. The PET apparatus uses coincidence counting to detect photon pairs with an energy of 511 keV, generated by electron-positron pair annihilation within a subject injected with a positron-emitting isotope (RI ray source), and collects this coincidence information. Based on this collected coincidence information, a tomographic image representing the spatial distribution of photon pair generation frequencies (i.e., the spatial distribution of the RI ray source) within the measurement space can be reconstructed.

[0004] In this case, the list data, which contains the coincidence count information collected by the PET device and is arranged in time series, is divided into multiple frames in the order in which it was collected. Image reconstruction processing is performed using the data contained in each frame of the list data, thereby obtaining a dynamic PET image composed of tomographic images from multiple frames. PET devices play an important role in the field of nuclear medicine, and can be used to study, for example, biological functions and higher-level brain functions.

[0005] Because the tomographic images reconstructed in this way contain a significant amount of noise, they must be removed using image filters. Examples of image filters used for noise removal include Gaussian filters and guided filters. Currently, Gaussian filters are used. In contrast, guided filters, developed in recent years, are characterized by their ability to better preserve the boundaries between deep and shallow areas in images compared to Gaussian filters.

[0006] Patent Document 1 and Non-Patent Documents 1 and 2 describe techniques for removing noise from dynamic PET images using guided filters. In the techniques described in Patent Document 1 and Non-Patent Document 1, an image obtained by integrating dynamic PET images composed of multiple frames of tomographic images is used as a guide image during noise removal processing using a guided filter. Furthermore, the technique described in Non-Patent Document 2 enables more effective noise removal by using a more appropriate guide image.

[0007] Non-Patent Document 3 describes a technique for removing noise from PET images by using the Deep Image Prior technique (Non-Patent Document 4) which is a type of deep neural network (DNN: Deep Neural Network) and a convolutional neural network (CNN: Convolutional Neural Network).

[0008] Prior art literature

[0009] Patent Literature

[0010] Patent Document 1: China Patent Application Publication No. 103955899

[0011] Non-patent literature

[0012] Non-patent document 1: Lijun Lu et al., "Dynamic PET Denoising Incorporating aComposite Image Guided Filter", IEEE Nuclear Science Symposium and MedicalImaging Conference (NSS / MIC), 2014

[0013] Non-patent document 2: F.Hashimoto et al., "Denoising of Dynamic Sinogram byImage Guided Filtering for Positron Emission Tomography", IEEE Transactions on Radiation and Plasma Medical Sciences, Vol.2No.6, pp.541-548, 2018

[0014] Non-Patent Literature 3: Kuang Gong et al., "PET Image Reconstruction Using DeepImage Prior", IEEE Transactions on Medical Imaging, 2018

[0015] Non-patent literature 4: Dmitry Ulyanov et al., "Deep Image Prior", arXiv preprint arXiv:1711.10925, 2017

[0016] Non-patent document 5: F.Hashimoto et al., "Dynamic PET Image Denoising UsingDeep Convolutional Neural Networks Without Prior Training Datasets", IEEEAccess, Vol.7, pp.96594-96603, 2019 Summary of the Invention

[0017] Problems to be solved by the invention

[0018] The noise removal performance of PET images using the techniques described in Patent Document 1 and Non-Patent Documents 1 to 3 is superior to that using a Gaussian filter. However, further improvement in noise removal performance is desired for PET and SPECT images.

[0019] A technique that can meet the demand for improved noise removal performance is described in Non-Patent Document 5. However, the noise removal technique described in Non-Patent Document 5 requires a long time for processing.

[0020] An object of the present invention is to provide an apparatus and method capable of generating a tomographic image with high-performance noise removal based on list data collected by a radiation tomography apparatus and shortening the time required for the noise removal process.

[0021] Means for solving problems

[0022] An embodiment of the present invention is an image processing device. An image processing device is a device that divides list data collected by a radiation tomography device into M frames in the order of collection, and performs a reconstruction process based on the list data included in the m-th frame for each m greater than or equal to 1 and less than or equal to M, thereby removing noise from an m-th tomographic image generated. The image processing device comprises: (1) a feature extraction unit that inputs an input image into a feature extraction neural network and outputs an intermediate image from the feature extraction neural network; (2) a reconstruction unit that inputs the intermediate image into an m-th reconstruction neural network for each m greater than or equal to 1 and less than or equal to M, and outputs an m-th output image from the m-th reconstruction neural network; (3) an evaluation unit that obtains an evaluation value based on the sum of differences between the m-th tomographic image and the m-th output image for each m greater than or equal to 1 and less than or equal to M; and (4) a control unit that repeats the processing of the feature extraction unit, the reconstruction unit, and the evaluation unit, as well as the learning of the feature extraction neural network and the m-th reconstruction neural network for each m greater than or equal to 1 and less than or equal to M based on the evaluation value, and outputs a plurality of m-th output images from the m-th reconstruction neural network for each m greater than or equal to 1 and less than or equal to M.

[0023] An embodiment of the present invention is a radiation tomography system. The radiation tomography system includes: a radiation tomography apparatus that collects list data for reconstructing a tomographic image of a subject; and an image processing apparatus having the above-described structure that generates a noise-removed tomographic image based on the list data collected by the radiation tomography apparatus.

[0024] An embodiment of the present invention is an image processing method. An image processing method is a method for removing noise from an m-th slice image generated by dividing list data collected by a radiation tomography apparatus into M frames in the order of collection, and performing a reconstruction process based on the list data included in the m-th frame for each m frame, the image processing method comprising: (1) a feature extraction step of inputting an input image into a feature extraction neural network and outputting an intermediate image from the feature extraction neural network; (2) a reconstruction step of inputting the intermediate image into an m-th reconstruction neural network for each m frame, and outputting an m-th output image from the m-th reconstruction neural network; (3) an evaluation step of obtaining an evaluation value based on the sum of differences between the m-th slice image and the m-th output image for each m frame; and (4) a learning step of learning each of the feature extraction neural network and the m-th reconstruction neural network for each m frame based on the evaluation value. The image processing method repeats the feature extraction step, the reconstruction step, the evaluation step, and the learning step, and outputs a plurality of m-th output images from the m-th reconstruction neural network for each m frame.

[0025] Effects of the Invention

[0026] According to the embodiment of the present invention, a tomographic image subjected to high-performance noise removal can be generated based on list data collected by a radiation tomography apparatus, and the time required for the noise removal process can be shortened. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a diagram showing the configuration of the radiation tomography system 1 .

[0028] Figure 2 1 is a diagram showing the configuration of a main portion of the image processing device 3 .

[0029] Figure 3 is a flowchart illustrating an image processing method.

[0030] Figure 4 This is a graph showing a model of temporal changes in activity of each of the white matter (WM), gray matter (GM), and tumor used in simulation.

[0031] Figure 5 It is a diagram showing a numerical phantom image.

[0032] Figure 6 is a diagram showing the 16th tomographic image x before noise removal processing. 16 Picture.

[0033] Figure 7 It shows the Figure 6 The 16th slice image x 16 This figure shows an image obtained by the noise removal process of Comparative Example 1.

[0034] Figure 8 It shows the Figure 6 The 16th slice image x 16 This figure shows an image obtained by the noise removal process of Comparative Example 2.

[0035] Figure 9 FIG. 1 is a diagram showing a tomographic image (static PET image) generated by performing reconstruction processing using all list data.

[0036] Figure 10 It shows the Figure 6 The 16th slice image x 16 This figure shows an image obtained by the noise removal process of Comparative Example 3.

[0037] Figure 11 It shows the Figure 6 The 16th slice image x 16 This figure shows an image obtained by performing the noise removal process of Example 1.

[0038] Figure 12 It is a figure showing an MRI image.

[0039] Figure 13 It shows the Figure 6 The 16th slice image x 16 This figure shows an image obtained by the noise removal process of Comparative Example 4.

[0040] Figure 14 It shows the Figure 6 The 16th slice image x 16 This figure shows an image obtained by performing the noise removal process of Example 2.

[0041] Figure 15 Graphs showing temporal changes in PSNR for tomographic images before noise removal, images after noise removal in Comparative Examples 1 to 4, and images after noise removal in Examples 1 and 2.

[0042] Figure 16 Graphs showing temporal changes in SSIM for tomographic images before noise removal processing, images after noise removal processing of Comparative Examples 1 to 4, and images after noise removal processing of Examples 1 and 2. DETAILED DESCRIPTION

[0043] Hereinafter, embodiments of the image processing apparatus and the image processing method will be described in detail with reference to the accompanying drawings. In the description of the drawings, identical elements are denoted by identical reference numerals, and duplicate descriptions are omitted. The present invention is not limited to these examples.

[0044] Figure 1 1 is a diagram showing a configuration of a radiation tomography system 1. The radiation tomography system 1 includes a radiation tomography apparatus 2 and an image processing apparatus 3. Figure 2 1 is a diagram showing the configuration of a main portion of the image processing device 3 .

[0045] The radiation tomography apparatus 2 is an apparatus that collects list data for reconstructing a tomographic image of a subject. Examples of the radiation tomography apparatus 2 include a PET apparatus and a SPECT apparatus. Hereinafter, the radiation tomography apparatus 2 will be described as a PET apparatus.

[0046] The image processing device 3 includes an image generation unit 11, a feature extraction unit 12, a reconstruction unit 13, an evaluation unit 14, a control unit 15, an image selection unit 16, and a storage unit 17. A computer including a CPU, RAM, ROM, a hard disk drive, and the like is used as the image processing device 3. Furthermore, the image processing device 3 includes an input unit (e.g., a keyboard, a mouse) for receiving input from the operator and a display unit (e.g., a liquid crystal display) for displaying images and the like.

[0047] The radiation tomography apparatus 2 includes a detection unit having multiple small radiation detectors arranged around a measurement space where a subject is placed. The radiation tomography apparatus 2 uses a coincidence counting method to detect photon pairs with an energy of 511 keV, generated by electron-positron pair annihilation within a subject injected with a positron-emitting isotope (RI ray source), and accumulates this coincidence count information. The radiation tomography apparatus 2 then outputs this accumulated coincidence count information, arranged in time series, as list data to the image processing apparatus 3.

[0048] The list data includes identification information and detection time information of a pair of radiation detectors that simultaneously counted photon pairs. The list data may further include energy information of photons detected by each radiation detector and detection time difference information of the pair of radiation detectors.

[0049] Image processing device 3 reconstructs a tomographic image based on the list data. Known techniques for reconstructing tomographic images based on list data include the ML-EM (maximum likelihood expectation maximization) method and a successive approximation image reconstruction technique using a modified block iteration method. Also known as successive approximation image reconstruction techniques using a block iteration method include the OSEM (ordered subset ML-EM) method, the RAMLA (row-action maximum likelihood algorithm) method, and the DRAMA (dynamic RAMLA) method. Furthermore, image processing device 3 uses a DNN (preferably a CNN) to generate a tomographic image after noise removal.

[0050] The image generator 11 divides the list data into a plurality of frames (first to Mth frames) in the order in which they were collected, and performs reconstruction processing on each of the plurality of frames using the data included in the mth frame in the list data to generate a tomographic image x of the mth frame. m (mth slice image x m ). M tomographic images x1~x M are dynamic PET images and are used as training images for the neural network.

[0051] The image generator 11 may also generate an input image z to be input to the neural network. The input image z is preferably an image representing the morphological information of the subject. Specifically, the input image z may be an image generated by using the ratio m The input image z may also be a static PET image.

[0052] The input image z may be generated for each frame, for all frames, or for several frames. The input image z may also be a tomographic image generated by performing a reconstruction process using all the list data. Furthermore, the input image z may be an MRI image of the subject or a CT image of the subject.

[0053] The feature extraction unit 12 inputs the input image z to the feature extraction neural network (feature extraction NN) 18, and outputs an intermediate image from the feature extraction NN 18. The reconstruction unit 13 inputs the intermediate image to the mth reconstruction neural network (mth reconstruction NN) 19. m , from the mth reconstruction using NN 19 m Output the mth output image y m,n Here, m is an integer greater than or equal to 1 and less than or equal to M. n is an integer greater than or equal to 0, and represents the feature extraction NN 18 and the m-th reconstruction NN 19. m The number of times of learning.

[0054] NN 18 for feature extraction and NN 19 for mth reconstruction m It is a DNN, preferably a CNN. The feature extraction NN 18 may also have a U-net structure. M reconstruction NNs 191 to 19 M It can also be a common structure.

[0055] The evaluation unit 14 evaluates the m-th tomographic image x m With the mth output image y m,n The evaluation unit 14 may calculate the evaluation value by, for example, the following equation (1), or, if the input image z is a static PET image, further calculate the evaluation value by using the following equation (2).

[0056]

[0057]

[0058] The control unit 15 repeats the processing of the feature extraction unit 12 and the reconstruction unit 13, the calculation of the evaluation value by the evaluation unit 14, and the calculation of the feature extraction NN 18 and the m-th reconstruction NN 19 based on the evaluation value. mThen, the control unit 15 performs n-times learning on the m-th reconstruction NN 19. m Output the mth output image y m,n In addition, feature extraction uses NN 18 and m-th reconstruction uses NN 19 m The respective learnings are performed together so that the evaluation values ​​become smaller.

[0059] The image selection unit 16 selects the image y from the plurality of m-th output images y for each of the first to M-th frames. m,0 ~y m,N Preferably, the image selection unit 16 selects an arbitrary image as the tomographic image after noise removal processing based on the m-th output image y m,n Comparison with the input image z, from multiple m-th output images y m,0 ~y m,N Select any mth output image from .

[0060] For example, you can also select the mth output image y m,n The error between the mth output image and the input image z is the smallest, and the mth output image y can also be obtained from the mth output image m,n The doctor or technician may select any m-th output image from one or more m-th output images whose error with the input image z is less than the threshold value. m,0 ~y m,N Select any mth output image from .

[0061] The storage unit 17 stores the list data, input image z, intermediate images, and the m-th output image y of each frame. m,n and the mth tomographic image x of each frame m In addition, the storage unit 17 stores a plurality of m-th output images y from each frame. m,0 ~y m,N The mth output image selected in .

[0062] Figure 3 1 is a flowchart illustrating an image processing method comprising an image generation step S1 , a feature extraction step S2 , a reconstruction step S3 , an end determination step S4 , an evaluation step S5 , a learning step S6 , and an image selection step S7 .

[0063] The image generation step S1 is a process performed by the image generation unit 11. In the image generation step S1, the list data is divided into a plurality of frames (1st to Mth frames) in the order in which they were collected. For each of the plurality of frames, a reconstruction process is performed using the data included in the mth frame of the list data to generate a tomographic image x of the mth frame. m (mth slice image x m) In addition, in the image generation step S1, for example, reconstruction processing may be performed using all the list data to generate a tomographic image (input image z).

[0064] The feature extraction step S2 is a process performed by the feature extraction unit 12. In the feature extraction step S2, the input image z is input to the feature extraction NN 18, and the feature extraction NN 18 outputs an intermediate image.

[0065] The reconstruction step S3 is a process performed by the reconstruction unit 13. In the reconstruction step S3, the intermediate image is input to the m-th reconstruction NN 19. m and reconstructed from the mth NN 19 m Output the mth output image y m,n .

[0066] The end determination step S4 is a process performed by the control unit 15. In the end determination step S4, it is determined whether the repetition of each process of steps S2 to S6 has ended. This determination can be based on whether the number of repetitions has reached a predetermined value or based on whether the m-th output image y m,n Is the error between the input image z and the image below the threshold?

[0067] In the end determination step S4 , if it is determined that the repetition of the processes of steps S2 to S6 is to be continued, the process proceeds to the evaluation step S5 . If it is determined that the repetition of the processes of steps S2 to S6 can be terminated, the process proceeds to the image selection step S7 .

[0068] The evaluation step S5 is a process performed by the evaluation unit 14. In the evaluation step S5, based on the m-th tomographic image x m With the mth output image y m,n The evaluation value is obtained by summing the differences between them (for example, the above formula (1) or (2)).

[0069] The learning step S6 is a process in which the control unit 15 performs a learning operation on the feature extraction NN 18 and the m-th reconstruction NN 19. m In the learning step S6, the feature extraction NN 18 and the m-th reconstruction NN 19 are simultaneously performed based on the evaluation value. m After the learning step S6, the process returns to the feature extraction step S2.

[0070] The image selection step S7 is a process performed by the image selection unit 16. In the image selection step S7, for each of the first to M-th frames, the image y is selected from the plurality of m-th output images. m,0 ~y m,N An arbitrary image is selected as the tomographic image after noise removal processing.

[0071] Next, the simulation results are explained. The numerical phantom used in this simulation is a simulation of 18 This is a model of the human brain using F-FDG (fluorodeoxyglucose). This digital membrane includes white matter (WM), gray matter (GM), and tumor.

[0072] Figure 4 This graph shows a model of the temporal changes in activity of the white matter (WM), gray matter (GM), and tumor used in the simulation. The time-activity curve (TAC) generally increases over time from the moment the RI radiation source is applied, peaks at a certain point, and then gradually decreases.

[0073] Set the total count (number of simultaneous counts) to 1×10 9 The measurement time is set to 90 minutes, and the list data is divided into 30 frames. The duration of each frame from 1 to 4 is set to 20 seconds, the duration of each frame from 5 to 8 is set to 40 seconds, the duration of each frame from 9 to 12 is set to 60 seconds, the duration of each frame from 13 to 16 is set to 180 seconds, and the duration of each frame from 17 to 30 is set to 300 seconds.

[0074] The digital volume mask has a three-dimensional structure of 192×192×64 voxels. Based on this digital volume mask, a sinogram is generated for each frame. The sinogram is a histogram of the coincidence count information for each pair of radiation detectors in the radiation tomography apparatus 2. Poisson noise corresponding to the count of each frame is added to the sinogram of each frame to generate a noise-added sinogram. Based on this noise-added sinogram, a reconstructed image (mth tomographic image x ) is generated using the OS-EM method. m ). The number of iterations in the OS-EM method is set to 6, and the number of subsets is set to 16.

[0075] In addition to the numerical volume membrane image, an example of a tomographic image before or after noise removal is shown below. In addition, each figure shows two tomographic images whose slice planes are orthogonal to each other. Figure 5 is a diagram showing a numerical body membrane image. Figure 5 In the image, the arrow indicates the tumor site. Figure 6 is a diagram showing the 16th tomographic image x before noise removal processing. 16 The Figure 6 The image has a lot of noise, making it difficult to identify the tumor area.

[0076] Figure 7It shows the Figure 6 The 16th slice image x 16 The image obtained by the noise removal process of Comparative Example 1 is shown in FIG. The noise removal process of Comparative Example 1 is a process using a Gaussian filter. Figure 7 Although the image is noisy and not clear overall, it can be used to identify the tumor site.

[0077] Figure 8 It shows the Figure 6 The 16th slice image x 16 The image obtained by the noise removal process of Comparative Example 2 is compared. The noise removal process of Comparative Example 2 is the process described in Non-Patent Document 2. Figure 8 Although the image is noisy, the boundaries between deep and shallow areas become clear, making it possible to identify the tumor area.

[0078] Figure 9 1 is a diagram showing a tomographic image (static PET image) generated by performing reconstruction processing using all list data. In Comparative Example 3 and Example 1, this static PET image is used as an input image to the neural network.

[0079] Figure 10 It shows the Figure 6 The 16th slice image x 16 The image obtained by the noise removal process of Comparative Example 3 is compared. The noise removal process of Comparative Example 3 is the process described in Non-Patent Document 5. Figure 10 Compared with the images of Comparative Examples 1 and 2, the noise in the image is sufficiently removed, the boundary between the deep and shallow parts is clear, and the tumor part can be identified.

[0080] Figure 11 It shows the Figure 6 The 16th slice image x 16 The image obtained by performing the noise removal process of Example 1. The noise removal process of Example 1 is a process according to this embodiment. Figure 11 Images and Figure 10 Compared with the image of , the noise is further fully removed, the boundary between deep and shallow becomes clear, and the tumor area can be identified.

[0081] Figure 12 : is a diagram showing an MRI image (T1-weighted image). In this MRI image, no tumor is observed. In Comparative Example 4 and Example 2, this MRI image is used as an input image to the neural network.

[0082] Figure 13 It shows the Figure 6 The 16th slice image x 16The image obtained by the noise removal process of Comparative Example 4 is shown in FIG. The noise removal process of Comparative Example 4 is the process described in Non-Patent Document 5. Figure 13 Compared with the images of Comparative Examples 1 and 2, the noise in the image is sufficiently removed, the boundary between the deep and shallow parts is clear, and the tumor part can be identified.

[0083] Figure 14 It shows the Figure 6 The 16th slice image x 16 The image obtained by performing the noise removal process of Example 2. The noise removal process of Example 2 is a process according to this embodiment. Figure 14 Images and Figure 13 Compared with the image of , the noise is further fully removed, the boundary between deep and shallow becomes clear, and the tumor area can be identified.

[0084] Figure 15 This graph shows the temporal changes in PSNR for tomographic images before noise removal, images after noise removal in Comparative Examples 1 to 4, and images after noise removal in Examples 1 and 2. PSNR (Peak Signal to Noise Ratio) indicates image quality in decibels (dB), with higher values ​​indicating better image quality.

[0085] Figure 16 This graph shows the temporal changes in SSIM for tomographic images before noise removal, images after noise removal from Comparative Examples 1 to 4, and images after noise removal from Examples 1 and 2. SSIM (Structural Similarity Index) quantifies changes in image brightness, contrast, and structure, with higher values ​​indicating better image quality.

[0086] Both the PSNR and SSIM indicators show that the noise removal processing of Comparative Examples 3 and 4 performs better than the noise removal processing of Comparative Examples 1 and 2. Furthermore, the noise removal processing of Examples 1 and 2 performs better than the noise removal processing of Comparative Examples 3 and 4. The noise removal processing of Examples 1 and 2 also performs well for other frames.

[0087] The reason why the noise removal process of this embodiment has superior performance compared to the noise removal process described in Non-Patent Document 5 is as follows.

[0088] The image processing device 3 of this embodiment includes processing by the feature extraction NN 18 of the feature extraction unit 12 and processing by the m-th reconstruction NN 19 of each frame of the reconstruction unit 13. mFurthermore, the image processing device 3 inputs the intermediate image output from the feature extraction NN 18 to the m-th reconstruction NN 19 of each frame. m , based on the reconstruction from the mth NN 19 m The output image y m,n With the mth slice image x m The evaluation value is obtained by summing up the differences between the two, and based on the evaluation value, the feature extraction NN 18 and the m-th reconstruction NN 19 are used. m Each study in .

[0089] Since the mth tomographic image x of each frame m Therefore, by repeating the above-mentioned learning based on the evaluation value, the feature extraction NN 18 can output information representing the m-th tomographic image x from each frame. m Extract the common information of the intermediate image and reconstruct the mth image using NN 19 m According to the intermediate image, the mth output image y is inferred m,n Therefore, it is considered that the performance of the noise removal process is improved in this embodiment.

[0090] In the noise removal process described in Non-Patent Document 5, the m-th tomographic image x of each frame is m A training image is used, and a neural network is trained using a combination of this training image and the input image z. This training is repeated for the number of frames. In contrast, in the noise removal process of this embodiment, the feature extraction NN 18 can be trained collectively for multiple frames. Therefore, compared to the noise removal process described in Non-Patent Document 5, the noise removal process of this embodiment can shorten the required time.

[0091] The present invention is not limited to the above-described embodiment and configuration examples, and various modifications are possible. For example, the radiation tomography apparatus 2 is a PET apparatus in the above-described embodiment, but may also be a SPECT apparatus.

[0092] The image processing device of the above-mentioned embodiment is a device for removing noise from an m-th tomographic image generated by dividing list data collected by a radiation tomography device into M frames in the order of collection, and performing a reconstruction process based on the list data included in the m-th frame for each m greater than or equal to 1 and less than or equal to M. The image processing device includes: (1) a feature extraction unit that inputs an input image into a feature extraction neural network and outputs an intermediate image from the feature extraction neural network; (2) a reconstruction unit that inputs the intermediate image into an m-th reconstruction neural network for each m greater than or equal to 1 and less than or equal to M, and outputs an m-th output image from the m-th reconstruction neural network; (3) an evaluation unit that obtains an evaluation value based on the sum of differences between the m-th tomographic image and the m-th output image for each m greater than or equal to 1 and less than or equal to M; and (4) a control unit that repeats the processing of the feature extraction unit, the reconstruction unit, and the evaluation unit, and the learning of the feature extraction neural network and the m-th reconstruction neural network for each m greater than or equal to 1 and less than or equal to M based on the evaluation value, and outputs a plurality of m-th output images from the m-th reconstruction neural network for each m greater than or equal to 1 and less than or equal to M.

[0093] In the above-mentioned image processing apparatus, the feature extraction unit may be configured to input an image representing morphological information of the subject as an input image to the feature extraction neural network. Furthermore, the feature extraction unit may be configured to input a tomographic image generated by performing a reconstruction process using more list data than the list data used when generating the m-th tomographic image for each m between 1 and M as an input image to the feature extraction neural network.

[0094] In the above-mentioned image processing apparatus, the feature extraction unit may be configured to input an MRI image of the subject into the feature extraction neural network. Alternatively, the feature extraction unit may be configured to input a CT image of the subject into the feature extraction neural network.

[0095] The image processing device may further include an image selection unit configured to select, for each m that is greater than or equal to 1 and less than or equal to M, an arbitrary m-th output image from the obtained plurality of m-th output images as the tomographic image after noise removal processing. Alternatively, the image selection unit may be configured to select, for each m that is greater than or equal to 1 and less than or equal to M, an arbitrary m-th output image from the plurality of m-th output images based on a comparison between the m-th output image and the input image. Alternatively, the image selection unit may be configured to select, for each m that is greater than or equal to 1 and less than or equal to M, an arbitrary m-th output image from the plurality of m-th output images based on a comparison between the m-th tomographic image and the input image.

[0096] The radiation tomography system according to the above embodiment is configured to include: a radiation tomography apparatus that collects list data for reconstructing a tomographic image of a subject; and an image processing apparatus having the above structure that generates a tomographic image after noise removal processing based on the list data collected by the radiation tomography apparatus.

[0097] The image processing method of the above embodiment is a method for removing noise from an m-th slice image generated by dividing list data collected by a radiation tomography apparatus into M frames in the order of collection, and performing a reconstruction process based on the list data included in the m-th frame for each m from 1 to M. The image processing method comprises: (1) a feature extraction step of inputting an input image into a feature extraction neural network and outputting an intermediate image from the feature extraction neural network; (2) a reconstruction step of inputting the intermediate image into an m-th reconstruction neural network for each m from 1 to M and outputting an m-th output image from the m-th reconstruction neural network; (3) an evaluation step of obtaining an evaluation value based on the sum of differences between the m-th slice image and the m-th output image for each m from 1 to M and M; and (4) a learning step of learning each of the feature extraction neural network and the m-th reconstruction neural network for each m from 1 to M based on the evaluation value. The image processing method repeats each of the feature extraction step, the reconstruction step, the evaluation step, and the learning step, and outputs a plurality of m-th output images from the m-th reconstruction neural network for each m from 1 to M.

[0098] In the above-described image processing method, the feature extraction step may be configured such that an image representing morphological information of the subject is input as an input image to the feature extraction neural network. Furthermore, the feature extraction step may be configured such that a tomographic image generated by reconstructing the image using more list data than the list data used when generating the m-th tomographic image for each m between 1 and M is input as an input image to the feature extraction neural network.

[0099] In the above-mentioned image processing method, the feature extraction step may be configured such that an MRI image of the subject is input as an image to the feature extraction neural network. Alternatively, the feature extraction step may be configured such that a CT image of the subject is input as an image to the feature extraction neural network.

[0100] The above-mentioned image processing method may be configured to further include an image selection step of selecting, for each m that is greater than or equal to 1 and less than or equal to M, an arbitrary m-th output image from the obtained plurality of m-th output images as the tomographic image after noise removal processing. Alternatively, the image selection step may be configured to select, for each m that is greater than or equal to 1 and less than or equal to M, an arbitrary m-th output image from the plurality of m-th output images based on a comparison between the m-th output image and the input image. Alternatively, the image selection step may be configured to select, for each m that is greater than or equal to 1 and less than or equal to M, an arbitrary m-th output image from the plurality of m-th output images based on a comparison between the m-th tomographic image and the input image.

[0101] Industrial applicability

[0102] The present invention can be used as an apparatus and method that can generate a tomographic image with high-performance noise removal based on list data collected by a radiation tomography apparatus and can shorten the time required for the noise removal process.

[0103] Explanation of symbols

[0104] 1...Radiation tomography system, 2...Radiation tomography apparatus, 3...Image processing apparatus, 11...Image generation unit, 12...Feature extraction unit, 13...Reconstruction unit, 14...Evaluation unit, 15...Control unit, 16...Image selection unit, 17...Storage unit, 18...Feature extraction neural network (NN for feature extraction), 19... m ...mth reconstruction neural network (mth reconstruction NN).

Claims

1. An image processing device, wherein: A device for removing noise from an m-th tomographic image generated by dividing list data collected by a radiation tomography apparatus into M frames in the order in which they were collected, and performing a reconstruction process based on the list data included in the m-th frame for each m number greater than or equal to 1 and less than or equal to M. The image processing device comprises: a feature extraction unit that inputs an input image into a feature extraction neural network and outputs an intermediate image from the feature extraction neural network; a reconstruction unit that inputs the intermediate image into an m-th reconstruction neural network for each m that is greater than or equal to 1 and less than or equal to M, and outputs an m-th output image from the m-th reconstruction neural network; an evaluation unit that obtains an evaluation value based on a total of differences between the m-th tomographic image and the m-th output image for each m number of images that is greater than or equal to 1 and less than or equal to M; and A control unit that repeats the processing of the feature extraction unit, the reconstruction unit, and the evaluation unit, and the learning of the feature extraction neural network and the m-th reconstruction neural network for each m greater than 1 and less than M based on the evaluation value, and outputs a plurality of m-th output images from the m-th reconstruction neural network for each m greater than 1 and less than M.

2. The image processing apparatus according to claim 1, wherein: The feature extraction unit inputs an image representing morphological information of a subject as the input image into the feature extraction neural network.

3. The image processing apparatus according to claim 1, wherein: The feature extraction unit inputs a tomographic image generated by reconstructing the image using more list data than list data used when generating the m-th tomographic image for each m greater than or equal to 1 and less than or equal to M as the input image to the feature extraction neural network.

4. The image processing apparatus according to claim 1, wherein: The feature extraction unit inputs the MRI image of the subject as the input image into the feature extraction neural network.

5. The image processing apparatus according to claim 1, wherein: The feature extraction unit inputs a CT image of a subject as the input image into the feature extraction neural network.

6. The image processing device according to any one of claims 1 to 5, wherein: The apparatus further includes an image selection unit configured to select, for each m not less than 1 and not more than M, an arbitrary m-th output image from the plurality of obtained m-th output images as a tomographic image after noise removal processing.

7. The image processing apparatus according to claim 6, wherein: The image selection unit selects an arbitrary m-th output image from the plurality of m-th output images based on a comparison between the m-th output image and the input image for each m that is greater than or equal to 1 and less than or equal to M.

8. A radiation tomography system, wherein: have: a radiation tomography apparatus that collects list data for reconstructing a tomographic image of a subject; and The image processing apparatus according to any one of claims 1 to 7, which generates a tomographic image after noise removal processing based on list data collected by the radiation tomography apparatus.

9. An image processing method, wherein: A method for removing noise from an m-th tomographic image generated by dividing list data collected by a radiation tomography apparatus into M frames in the order of collection, and performing a reconstruction process based on the list data included in the m-th frame for each m greater than or equal to 1 and less than or equal to M. The image processing method comprises: a feature extraction step of inputting an input image into a feature extraction neural network and outputting an intermediate image from the feature extraction neural network; a reconstruction step of inputting the intermediate image into an m-th reconstruction neural network for each m that is greater than or equal to 1 and less than or equal to M, and outputting an m-th output image from the m-th reconstruction neural network; an evaluation step of obtaining an evaluation value based on a sum of differences between the m-th tomographic image and the m-th output image for each m number of images, which is greater than or equal to 1 and less than or equal to M; and A learning step of causing the feature extraction neural network and each of the m-th reconstruction neural networks of m being greater than or equal to 1 and less than or equal to M to learn based on the evaluation value. The feature extraction step, the reconstruction step, the evaluation step, and the learning step are repeatedly performed, and a plurality of m-th output images are output from the m-th reconstruction neural network for each m number greater than or equal to 1 and less than or equal to M.

10. The image processing method according to claim 9, wherein: In the feature extraction step, an image representing morphological information of a subject is input as the input image to the feature extraction neural network.

11. The image processing method according to claim 9, wherein: In the feature extraction step, a tomographic image generated by reconstructing the image using more list data than the list data used when generating the m-th tomographic image for each m greater than or equal to 1 and less than or equal to M is input as the input image to the feature extraction neural network.

12. The image processing method according to claim 9, wherein: In the feature extraction step, an MRI image of a subject is input as the input image to the feature extraction neural network.

13. The image processing method according to claim 9, wherein: In the feature extraction step, a CT image of a subject is input as the input image to the feature extraction neural network.

14. The image processing method according to any one of claims 9 to 13, wherein: The method further includes an image selection step of selecting, for each m not less than 1 and not more than M, an arbitrary m-th output image from the plurality of obtained m-th output images as a tomographic image after noise removal processing.

15. The image processing method according to claim 14, wherein: In the image selection step, for each m not less than 1 and not more than M, an arbitrary mth output image is selected from a plurality of the mth output images based on a comparison between the mth output image and the input image.

Images