Image processor and image processing method

The image processing device and method enhance tomographic image reconstruction by using a CNN with error evaluation terms and block-divided sinograms, addressing the inefficiency of existing DIP technology to achieve faster and more effective noise reduction.

JP2024157164APending Publication Date: 2024-11-07HAMAMATSU PHOTONICS KK
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2023071331
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-25
Publication Date
2024-11-07

AI Technical Summary

Technical Problem

Existing image processing methods using Deep Image Prior (DIP) technology for tomographic image reconstruction require a large number of CNN learning times, making it difficult to obtain a tomographic image with reduced noise in a timely manner.

Method used

An image processing device and method that utilizes a convolutional neural network (CNN) with an evaluation function to reduce the number of learning iterations by incorporating error evaluation terms between reconstructed and output images, and further divides sinograms into blocks for enhanced noise reduction.

Benefits of technology

The method significantly reduces the number of CNN learning times and achieves tomographic images with reduced noise in a shorter time frame.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024157164000001_ABST
    Figure 2024157164000001_ABST
Patent Text Reader

Abstract

To provide an image processor capable of, when creating a tomographic image of an analyte by making learning of a CNN on the basis of the evaluation result of an error between a calculated sinogram and an actually measured sinogram, obtaining the noise-reduced tomographic image in a short period of time by reducing the CNN learning frequency by the DIP technology.SOLUTION: An image processor 10 comprises a sinogram creation part 11, a CNN processing part 12, a convolutional integration part 13, a forward projection calculation part 14, a CNN learning part 15, a storage part 16 and an image reconstruction part 17. The forward projection calculation part 14 calculates the forward projection of an output image 23 to create a calculated sinogram 24. In a first processing step, the CNN learning part 15 makes learning of a CNN on the basis of an error between a reconstructed image created by the image reconstruction part 17 and the output image 23. In a following second processing step, the CNN learning part 15 makes learning of the CNN on the basis of an error between an actually measured sinogram 21 and the calculated sinogram 24.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an apparatus and method for producing a tomographic image of a subject based on coincidence counting information collected by a radiation tomography apparatus. [Background technology]

[0002] 2. Description of the Related Art Radiation tomography apparatus capable of acquiring a tomographic image of a subject (living body) includes a PET (Positron Emission Tomography) apparatus and a SPECT (Single Photon Emission Computed Tomography) apparatus.

[0003] A PET device is equipped with a detection section that has many small radiation detectors arranged around a measurement space in which a subject is placed. In a PET device, photon pairs with an energy of 511 keV are generated in association with the annihilation of electrons and positrons in a subject to which a positron-emitting isotope (RI source) has been administered, and the PET device collects the coincidence information by detecting the photon pairs with an energy of 511 keV using the coincidence method. Then, based on the collected coincidence information, a tomographic image can be reconstructed that shows the spatial distribution of the occurrence frequency of photon pairs in the measurement space (i.e., the spatial distribution of the RI source). This PET device plays an important role in the field of nuclear medicine, and can be used to study, for example, biological functions and higher-level brain functions.

[0004] Various methods are known as a method for reconstructing a tomographic image of a subject based on a large number of collected coincidence counting information. The image processing method for reconstructing a tomographic image described in Non-Patent Document 1 reconstructs a tomographic image by a Deep Image Prior technique using a convolutional neural network, which is a type of deep neural network. Hereinafter, the Convolutional Neural Network is referred to as "CNN" and the Deep Image Prior technique is referred to as "DIP technique." The DIP technique utilizes the property of CNN that meaningful structures in an image are learned faster than random noise (i.e., random noise is difficult to learn). The DIP technique makes it possible to obtain a tomographic image with reduced noise.

[0005] Specifically, the image processing method described in Non-Patent Document 1 is as follows: A sinogram (hereinafter referred to as a "measured sinogram") is created based on a large amount of coincidence information collected about a subject. In addition, when an input image (e.g., an MRI image) is input to a CNN, an image output from the CNN is subjected to forward projection calculation (Radon transform) to create a sinogram (hereinafter referred to as a "calculated sinogram"). Then, the error between this calculated sinogram and the measured sinogram is evaluated, and the CNN is trained based on the error evaluation result. By repeating the image output from the CNN, the creation of a calculated sinogram by forward projection calculation, the error evaluation, and the training of the CNN using the DIP technique, the calculated sinogram gradually approaches the measured sinogram, and the output image from the CNN approaches a tomographic image of the subject.

[0006] This image processing method includes a process of forward projection from a CNN output image to a calculated sinogram, but does not include a process of back projection from a measured sinogram to a tomographic image, so that a tomographic image with reduced noise can be obtained. In addition, the image processing method described in Non-Patent Document 1 divides each of the measured sinogram and the calculated sinogram into multiple blocks, evaluates the error between the calculated sinogram and the measured sinogram for each block, and trains the CNN, thereby reducing the memory capacity required during processing.

[0007] A sinogram is a histogram that represents the frequency with which coincidence information was acquired (frequency of occurrence of coincidence events) in a space (sinogram space) represented by four variables r, θ, z, and δ. The variable r represents the radial position of the coincidence line (the line connecting the two detectors that simultaneously counted photon pairs). The variable θ represents the azimuth angle of the coincidence line. The variable z represents the central axial position of the midpoint of the coincidence line. Additionally, the variable δ represents the polar angle of the coincidence line. [Prior art documents] [Non-patent literature]

[0008] [Non-Patent Document 1] F. Hashimoto, Y. Onishi, K. Ote, H. Tashima and T. Yamaya, "Fully 3D Implementation of the End-to-end DeepImage Prior-based PET Image Reconstruction Using Block IterativeAlgorithm," arXiv preprint, arXiv:2212.11844, 2022, doi:10.48550 / arXiv.2212.11844 [Non-Patent Document 2] J. Nuyts et al. "A concaveprior penalizing relative differences for maximum-a-posteriori reconstructionin emission tomography," IEEE TNS. 49:1, 56-60, 2002. [Non-Patent Document 3] Hiroshi Kudo, "Image reconstruction methods for low-dose CT - Statistical image reconstruction, iterative image reconstruction, and compressed sensing," Medical ImagingTechnology 32.4 (2014): 239-248. [Non-Patent Document 4] Chambolle, Antonin. "Analgorithm for total variation minimization and applications," Journal ofMathematical imaging and vision 20.1 (2004): 89-97. Summary of the Invention [Problem to be solved by the invention]

[0009] The image processing method using the DIP technique described in Non-Patent Document 1 has excellent noise reduction performance, but requires many CNN training rounds and takes a long time to obtain a tomographic image of the subject with reduced noise.

[0010] The present invention has been made to solve the above problems, and aims to provide an image processing device and an image processing method that, when training a CNN based on the evaluation results of the error between a calculated sinogram and an actual sinogram to create a tomographic image of a subject, can reduce the number of times CNN is trained using DIP technology, thereby enabling to obtain a tomographic image with reduced noise in a short time. [Means for solving the problem]

[0011] The image processing device of the present invention is an image processing device that creates a tomographic image of a subject based on coincidence counting information collected by a radiation tomography device having a plurality of detectors arranged surrounding a measurement space in which a subject administered with an RI radiation source is placed.

[0012] A first aspect of the image processing device of the present invention includes (1) an image reconstruction unit that creates a reconstructed image by an analytical image reconstruction method or a statistical image reconstruction method based on coincidence count information collected by a radiation tomography device, (2) a sinogram creation unit that creates a sinogram based on the coincidence count information collected by the radiation tomography device, (3) a CNN processing unit that inputs an input image to a convolutional neural network and creates an output image by the convolutional neural network, (4) a forward projection calculation unit that performs a forward projection calculation of the output image to create a sinogram, and (5) a CNN learning unit that trains the convolutional neural network.

[0013] In the first processing step, (a) the CNN learning unit uses an evaluation function including an error evaluation term that represents an evaluation value regarding the error between the reconstructed image and the output image, and trains the convolutional neural network based on the value of this evaluation function, and (b) repeats the processing of each of the CNN processing unit and the CNN learning unit multiple times. In the second processing step following the first processing step, (a) the CNN learning unit uses an evaluation function including an error evaluation term that represents an evaluation value regarding the error between the sinogram created by the sinogram creation unit and the sinogram created by the forward projection calculation unit, and trains the convolutional neural network based on the value of this evaluation function, and (b) the output image after the processing of each of the CNN processing unit, the forward projection calculation unit, and the CNN learning unit is repeated multiple times is a tomographic image of the subject.

[0014] The image processing device of the present invention may have the following configuration. In a second aspect of the image processing device of the present invention, in addition to the first aspect, the sinogram creation unit creates a sinogram divided into a plurality of blocks based on coincidence counting information collected by a radiation tomography device, the forward projection calculation unit performs forward projection calculation on the output image to create a sinogram divided into a plurality of blocks, and in a second processing step, the CNN learning unit trains a convolutional neural network based on the value of the evaluation function for each of the plurality of blocks.

[0015] In a third aspect of the image processing device of the present invention, in addition to the first or second aspect, each of the tomographic image, the reconstructed image, the input image, and the output image is a three-dimensional image.

[0016] A fourth aspect of the image processing device of the present invention, in addition to any of the first to third aspects, further includes a convolution integral unit that performs a convolution integral of a point spread function on an output image, and a forward projection calculation unit that performs a forward projection calculation of the output image after processing by the convolution integral unit.

[0017] In a fifth aspect of the image processing device of the present invention, in addition to any of the first to fourth aspects, in the second processing step, the CNN learning unit evaluates an error using an error evaluation term in an area in the sinogram space where coincidence information can be collected by the radiation tomography device.

[0018] In a sixth aspect of the image processing device of the present invention, in addition to any one of the first to fifth aspects, the CNN learning unit uses an evaluation function further including a regularization term representing an evaluation value related to a difference in pixel values ​​between adjacent pixels in an output image, and trains the convolutional neural network based on the value of this evaluation function.

[0019] In a seventh aspect of the image processing device of the present invention, in addition to any of the first to sixth aspects, the CNN processing unit inputs an image representing morphological information of the subject, an MRI image of the subject, a CT image of the subject, a static PET image of the subject, or a random noise image as an input image to the convolutional neural network.

[0020] The radiation tomography system of the present invention includes a radiation tomography device having a plurality of detectors arranged surrounding a measurement space in which a subject administered with an RI source is placed and which collects coincidence counting information, and the image processing device of the present invention described above which creates a tomographic image of the subject based on the coincidence counting information collected by the radiation tomography device.

[0021] The image processing method of the present invention is an image processing method for creating a tomographic image of a subject based on coincidence counting information collected by a radiation tomography apparatus having a plurality of detectors arranged surrounding a measurement space in which a subject administered with an RI radiation source is placed.

[0022] A first aspect of the image processing method of the present invention includes: (1) an image reconstruction step of creating a reconstructed image by an analytical image reconstruction method or a statistical image reconstruction method based on coincidence count information collected by a radiation tomography apparatus; (2) a sinogram creation step of creating a sinogram based on the coincidence count information collected by the radiation tomography apparatus; (3) a CNN processing step of inputting an input image to a convolutional neural network and creating an output image by the convolutional neural network; (4) a forward projection calculation step of performing a forward projection calculation on the output image to create a sinogram; and (5) a CNN learning step of training the convolutional neural network.

[0023] In the first processing step, (a) the CNN learning step uses an evaluation function including an error evaluation term that represents an evaluation value regarding the error between the reconstructed image and the output image, and trains a convolutional neural network based on the value of this evaluation function, and (b) the CNN processing step and the CNN learning step are each repeated multiple times. In the second processing step following the first processing step, (a) the CNN learning step uses an evaluation function including an error evaluation term that represents an evaluation value regarding the error between the sinogram created in the sinogram creation step and the sinogram created in the forward projection calculation step, and trains a convolutional neural network based on the value of this evaluation function, and (b) the output image after the CNN processing step, the forward projection calculation step, and the CNN learning step are each repeated multiple times is a tomographic image of the subject.

[0024] The image processing method of the present invention may be embodied as follows. In a second aspect of the image processing method of the present invention, in addition to the first aspect, the sinogram creation step creates a sinogram divided into a plurality of blocks based on coincidence counting information collected by a radiation tomography apparatus, the forward projection calculation step performs forward projection calculation on the output image to create a sinogram divided into a plurality of blocks, and in the second processing step, the CNN learning step trains a convolutional neural network based on the value of the evaluation function for each of the plurality of blocks.

[0025] In a third aspect of the image processing method of the present invention, in addition to the first or second aspect, each of the tomographic image, the reconstructed image, the input image and the output image is a three-dimensional image.

[0026] A fourth aspect of the image processing method of the present invention, in addition to any one of the first to third aspects, further includes a convolution integral step of performing a convolution integral of a point spread function on an output image, and a forward projection calculation step performs a forward projection calculation on the output image after processing by the convolution integral step.

[0027] In a fifth aspect of the image processing method of the present invention, in addition to any of the first to fourth aspects, in the second processing step, the CNN learning step evaluates an error using an error evaluation term in a region in the sinogram space where coincidence information can be collected by the radiation tomography apparatus.

[0028] In a sixth aspect of the image processing method of the present invention, in addition to any one of the first to fifth aspects, the CNN learning step uses an evaluation function further including a regularization term representing an evaluation value related to a difference in pixel values ​​between adjacent pixels in the output image, and trains the convolutional neural network based on the value of this evaluation function.

[0029] In a seventh aspect of the image processing method of the present invention, in addition to any one of the first to sixth aspects, the CNN processing unit inputs an image representing morphological information of the subject, an MRI image of the subject, a CT image of the subject, a static PET image of the subject, or a random noise image as an input image to the convolutional neural network. Effect of the Invention

[0030] According to the present invention, when creating a tomographic image of a subject by training a CNN based on the evaluation results of the error between a calculated sinogram and an actual sinogram, the number of times CNN is trained using DIP technology can be reduced, and a tomographic image with reduced noise can be obtained in a short time. [Brief description of the drawings]

[0031] [Figure 1] FIG. 1 is a diagram showing the configuration of a radiation tomography system 1. As shown in FIG. [Diagram 2] FIG. 2 is a diagram illustrating an example of the configuration of a CNN. [Diagram 3] FIG. 3 is a flow chart of an image processing method. [Figure 4] FIG. 4 is a flow chart of a first processing step of the image processing method. [Diagram 5] FIG. 5 is a flow chart of a second processing step of the image processing method. [Figure 6]Fig. 6 is a diagram showing a comparison between examples of calculated sinogram 24 when no block division is performed and calculated sinograms 241 to 2416 when block division is performed. Fig. 6(a) shows a schematic diagram of calculated sinogram 24 when no block division is performed. Fig. 6(b) shows a schematic diagram of calculated sinograms 241 to 2416 when block division is performed. [Figure 7] FIG. 7 is a diagram illustrating adjacent pixels in an output image. [Figure 8] FIG. 8 is a diagram showing an original image (correct image) of a tomographic image of a brain. [Figure 9] FIG. 9 shows a tomographic image of the brain obtained by image processing method 1. As shown in FIG. [Figure 10] FIG. 10 shows a tomographic image of the brain obtained by image processing method 2. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] Hereinafter, the embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are given the same reference numerals, and duplicated descriptions are omitted. The present invention is not limited to these examples, but is indicated by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.

[0033] 1 is a diagram showing the configuration of a radiation tomography system 1. The radiation tomography system 1 includes a radiation tomography apparatus 2 and an image processing device 10. The image processing device 10 includes a sinogram creation unit 11, a CNN processing unit 12, a convolution integral unit 13, a forward projection calculation unit 14, a CNN learning unit 15, a storage unit 16, and an image reconstruction unit 17.

[0034] The input image, reconstructed image, output image, and tomographic image handled by the image processing device 10 may be either two-dimensional images or three-dimensional images, but the following description assumes that these images are three-dimensional images. Also, the measured sinogram and the calculated sinogram may or may not be divided into multiple blocks, but the following description mainly focuses on the case where these sinograms are divided into multiple blocks.

[0035] The radiation tomography apparatus 2 is an apparatus that collects coincidence information for reconstructing a tomographic image of a subject. Examples of the radiation tomography apparatus 2 include a PET apparatus and a SPECT apparatus. In the following, the radiation tomography apparatus 2 will be described as a PET apparatus.

[0036] The radiation tomography apparatus 2 is provided with a detection unit having a large number of small radiation detectors arranged around a measurement space in which a subject is placed. The radiation tomography apparatus 2 detects photon pairs with energy of 511 keV generated in association with annihilation of electrons and positrons in a subject administered with an RI radiation source by a coincidence method using the detection unit, and collects this coincidence information. The radiation tomography apparatus 2 then outputs this collected coincidence information to the image processing device 10.

[0037] The image processing device 10 includes a GPU (Graphics Processing Unit) that performs processing using a convolutional neural network (CNN), an input unit (e.g., a keyboard or a mouse) that receives input from an operator, a display unit (e.g., a liquid crystal display) that displays images, etc., and a storage unit that stores programs and data for executing various processes. A computer having a CPU, RAM, ROM, a hard disk drive, etc. is used as the image processing device 10.

[0038] The memory unit 16 inputs and stores the coincidence information collected by the radiation tomography apparatus 2. The image reconstruction unit 17 creates a reconstructed image by a general image reconstruction method based on the coincidence information stored in the memory unit 16. The general image reconstruction method is not a reconstruction method using CNN, but an analytical image reconstruction method or a statistical image reconstruction method. An example of the analytical image reconstruction method is Filtered Back Projection (FBP), etc. An example of the statistical image reconstruction method is a method such as ML-EM (Maximum Likelihood Expectation Maximization), OSEM (Ordered Subset Expectation Maximization), RAMLA (Row Action Maximum Likelihood Algorithm), and DRAMA (Dynamic RAMLA), etc.

[0039] The sinogram creation unit 11 creates the measured sinogram 21 based on the coincidence counting information stored in the storage unit 16. At this time, the sinogram creation unit 11 creates the measured sinogram 21 divided into a plurality of (K) blocks. 1 ~21 K Create a measured sinogram 21 k is a measured sinogram of the kth block among the K blocks. K is an integer of 2 or more, and k is an integer of 1 or more and K or less. 1 ~21 K The combination of these forms the entire measured sinogram 21.

[0040] The CNN processing unit 12 inputs a three-dimensional input image 20 to the CNN, and uses the CNN to create a three-dimensional output image 22. The three-dimensional input image 20 may be an image representing morphological information of a subject, an MRI image, a CT image, or a static PET image of the subject, or a random noise image.

[0041] The convolution integral unit 13 performs convolution integral of a point spread function on the three-dimensional output image 22 created by the CNN processing unit 12 to create a new three-dimensional output image 23. The point spread function (PSF) is a function that represents the response (impulse response) of a radiation tomography apparatus to a point radiation source, and is generally represented by a Gaussian function or an asymmetric Gaussian function that is modeled from actual measurement data of the point radiation source and has a different blur depending on the position in the field of view. By providing the convolution integral unit 13, it is possible to obtain a tomographic image with better image quality and to stabilize the learning of the CNN.

[0042] The forward projection calculation unit 14 performs forward projection calculation on the three-dimensional output image 23 to generate a calculated sinogram 24. At this time, the forward projection calculation unit 14 calculates the calculated sinogram 24 divided into K blocks. 1 ~24 K Create a computational sinogram 24 k is the calculated sinogram of the kth block of the K blocks. 1 ~24 K The combination of these is the entire calculated sinogram 24.

[0043] The block division of the calculated sinogram 24 is performed in the same manner as the block division of the measured sinogram 21. k and the measured sinogram of the kth block 21 k is a sinogram of a common region of the entire sinogram space. The manner of block division is arbitrary, and the sinogram space may be divided into blocks for one or more of the four variables expressing the sinogram space. The sizes of the K blocks may be different or the same.

[0044] The CNN training unit 15 trains the CNN by the DIP technique using an evaluation function including an error evaluation term that differs between a first processing step and a second processing step described below. In the first processing step, the CNN training unit 15 uses an evaluation function including an error evaluation term that represents an evaluation value regarding the error between the reconstructed image created by the image reconstruction unit 17 and the output image created by the convolution integration unit 13, and trains the CNN based on the error evaluation result. In the second processing step, the CNN training unit 15 trains the CNN based on the measured sinogram 21 for each of the K blocks. k and Calculated Sinogram 24 k and trains the CNN based on the error evaluation results for each of the K blocks by using an error function including an error evaluation term that represents an evaluation value regarding the error between K and the K blocks. The CNN training unit 15 trains the CNN in the second processing step following the training of the CNN in the first processing step.

[0045] In the first processing step, the processes of the CNN processing unit 12, the convolution integration unit 13, and the CNN learning unit 15 are repeated multiple times. The processing flow of the first processing step is indicated by dashed arrows in the image processing device 10 in FIG.

[0046] In a second processing step following the first processing step, the CNN processing unit 12, the convolution integral unit 13, the forward projection calculation unit 14, and the CNN learning unit 15 perform the processes multiple times. The processing flow of the second processing step is indicated by solid arrows in the image processing device 10 in FIG. 1. The three-dimensional output image 22 created by the CNN processing unit 12 is the three-dimensional tomographic image of the subject. The three-dimensional output image 23 created by the convolution integral unit 13 may be the three-dimensional tomographic image of the subject. Since the measured sinogram 21 reflects the response function of the radiation tomography apparatus, it is preferable to use the three-dimensional output image 22 before the convolution integral of the point spread function by the convolution integral unit 13 as the three-dimensional tomographic image of the subject.

[0047] The convolution integration unit 13 may be provided as the final layer of the CNN, or may be provided separately from the CNN. When the convolution integration unit 13 is provided as the final layer of the CNN, the weighting coefficient of the convolution integration unit 13 is kept constant during the learning of the CNN.

[0048] Also, the convolution integral unit 13 may not be provided. When the convolution integral unit 13 is not provided, in the first processing step, the CNN learning unit 15 uses an evaluation function including an error evaluation term that represents an evaluation value regarding the error between the reconstructed image created by the image reconstruction unit 17 and the 3D output image 22 created by the CNN processing unit 12, and trains the CNN based on the error evaluation result. When the convolution integral unit 13 is not provided, in the second processing step, the forward projection calculation unit 14 performs a forward projection calculation on the 3D output image 22 output from the CNN processing unit 12 to create a calculated sinogram 24.

[0049] Fig. 2 is a diagram showing an example of the configuration of a CNN. The CNN shown in this figure has a three-dimensional U-net structure including an encoder and a decoder. In this figure, the size of each layer of the CNN is shown, assuming that the number of pixels of the three-dimensional input image 20 input to the CNN is N×N×64.

[0050] 3 to 5 are flowcharts of an image processing method. As shown in Fig. 3, the image processing method includes a first processing step S1 followed by a second processing step S2. The CNN training unit 15 trains the CNN in the first processing step, and subsequently trains the CNN in the second processing step as well.

[0051] As shown in FIG. 4, the first processing step S1 includes an image reconstruction step S17 performed by the image reconstruction unit 17, a CNN processing step S12 performed by the CNN processing unit 12, a convolution integration step S13 performed by the convolution integration unit 13, and a CNN learning step S15 performed by the CNN learning unit 15.

[0052] In an image reconstruction step S17, a reconstructed image is created using a general image reconstruction method. In a CNN processing step S12, a three-dimensional input image 20 is input to a CNN, which creates a three-dimensional output image 22. In a convolution integral step S13, a convolution integral of a point spread function is performed on the three-dimensional output image 22 created in the CNN processing step S12, to create a new three-dimensional output image 23.

[0053] In the CNN learning step S15, an evaluation function including an error evaluation term that represents an evaluation value regarding the error between the reconstructed image created in the image reconstruction step S17 and the 3D output image 23 created in the convolution integration step S13 is used, and the CNN is trained based on the error evaluation result. In the first processing step S1, the CNN processing step S12, the convolution integration step S13, and the CNN learning step S15 are each repeated multiple times.

[0054] As shown in FIG. 5, the second processing step S2 includes a sinogram creation step S21 performed by the sinogram creation unit 11, a CNN processing step S22 performed by the CNN processing unit 12, a convolution integration step S23 performed by the convolution integration unit 13, a forward projection calculation step S24 performed by the forward projection calculation unit 14, and a CNN learning step S25 performed by the CNN learning unit 15.

[0055] In the sinogram creation step S21, a measured sinogram 21 divided into K blocks is created based on coincidence information collected by the radiation tomography device 2. 1 ~21 K In CNN processing step S22, the 3D input image 20 is input to the CNN, and the CNN creates a 3D output image 22. In convolution integration step S23, the 3D output image 22 created in CNN processing step S22 is convoluted with the point spread function to create a new 3D output image 23.

[0056] In the forward projection calculation step S24, the three-dimensional output image 23 is forward projected to obtain a calculated sinogram 24 divided into K blocks. 1 ~24 K In the CNN learning step S25, the measured sinogram 21 is created for each of the K blocks. k and Calculated Sinogram 24 k The error between is evaluated, and the CNN is trained based on the error evaluation results for each of the K blocks.

[0057] In the second processing step S2, the CNN processing step S22, the convolution step S23, the forward projection calculation step S24, and the CNN learning step S25 are each repeated a number of times, and then the 3D output image 22 created in the CNN processing step S22 is set as the 3D tomographic image of the subject. The 3D output image 23 created in the convolution step S23 may be set as the 3D tomographic image of the subject. Note that the convolution step S23 may not be provided.

[0058] Next, the processing contents of each step of the image processing method when the sinogram is not divided into multiple blocks in the second processing step will be described. In the image processing method when the sinogram is not divided into blocks, the actual sinogram and the calculated sinogram are each processed as a whole.

[0059] In the following, the processing by CNN is denoted as f, the 3D input image 20 input to the CNN as z, and the weighting coefficient parameter representing the learning state of the CNN as θ. θ changes as the learning of the CNN progresses. The 3D output image 22 output from the CNN when a 3D input image z is input to a CNN with a weighting coefficient of θ is denoted as x. The 3D output image x is expressed by (1) below. In the CNN processing step, the processing represented by this formula is carried out to create the 3D output image x.

[0060]

number

[0061] In the convolution step, the point spread function is convolved with the 3D output image x created in the CNN processing step to create a new 3D output image x. In Figure 1, the 3D output image x after the convolution is denoted as PSF(f(θ|z)).

[0062] In the forward projection calculation step, the 3D output image x is forward projected to create a calculated sinogram 24. The calculated sinogram 24 is denoted as y, and the projection matrix for performing the forward projection calculation (Radon transform) from the 3D output image x to the calculated sinogram y is denoted as P. The projection matrix is ​​also called the system matrix or detection probability. The process performed in the forward projection calculation step is expressed by the following equation (2).

[0063]

number

[0064] In the CNN learning step in the second processing step, the measured sinogram 21 is 0 As the measured sinogram y 0 and the calculated sinogram y (equation (2) above), and the CNN is trained based on the error evaluation result. The process performed in the CNN training step is expressed by the following equation (3). The constrained optimization problem of this equation is solved by solving the evaluation function E(y;y 0 The problem is to optimize the CNN parameter θ so that the value of

[0065]

number

[0066] The constrained optimization problem of equation (3) can be transformed into an unconstrained optimization problem of equation (4) below. Any evaluation function E can be used, but for example, the L1 norm, the L2 norm, or the negative log-likelihood in the Poisson distribution can be used. When the L2 norm is used as the evaluation function, equation (4) can be transformed into equation (5) below.

[0067]

number

[0068]

number

[0069] Considering the arrangement of multiple detectors in a radiation tomography apparatus, there may be areas in the sinogram space where it is impossible to collect coincidence counting information. For this reason, the optimization problem of the above formula (5) may be replaced by the optimization problem of the following formula (6). In formula (6), m is a binary mask function, which has a value of 1 in areas in the sinogram space where it is possible to collect coincidence counting information, and a value of 0 in areas where it is impossible to collect coincidence counting information. Formula (6) is a function of the error (yy 0 ) and a binary mask function m, the error is selectively evaluated in a region in the sinogram space where coincidence information can be collected.

[0070]

number

[0071] In the second processing step, the CNN processing step, the convolution integral step, the forward projection calculation step, and the CNN learning step are each repeated multiple times to solve this optimization problem for the CNN parameter θ, thereby obtaining a calculated sinogram y that is equal to the measured sinogram y 0 , and the 3D output image x created by CNN approaches a tomographic image of the subject.

[0072] Next, the processing contents of each step of the image processing method when the sinogram is divided into blocks in the second processing step will be described in detail. When the sinogram is divided into blocks, in the forward projection calculation step, the three-dimensional output image x is forward projected to generate a calculated sinogram 24 divided into K blocks. 1 ~24 K Create the kth block's calculated sinogram 24 k y k Then, the sinogram y is calculated from the 3D output image x. k The projection matrix for forward projection calculation (Radon transform) to P k The process performed in the forward projection calculation step is expressed by the following equation (7).

[0073]

number

[0074] In the CNN learning step of the second processing step, the measured sinogram 21 of the kth block k y 0k Then, for each of the K blocks, the measured sinogram y 0k and the calculated sinogram y k The error between is evaluated, and the CNN is trained based on the error evaluation results for each of the K blocks. The process performed in the CNN training step is expressed as an unconstrained optimization problem in the following equation (8). If the L2 norm is used as the evaluation function, equation (8) can be transformed into equation (9) below. In addition, when the error is selectively evaluated in an area in the sinogram space where coincidence information can be collected, it is expressed as an unconstrained optimization problem in the following equation (10). m k is the binary mask function in the kth block.

[0075]

number

[0076]

number

[0077]

number

[0078] In the second processing step, the CNN processing step, the convolution integral step, the forward projection calculation step, and the CNN learning step are each repeated multiple times to solve the optimization problem for the CNN parameter θ, thereby calculating a sinogram y k is the measured sinogram y 0k , and the 3D output image x created by CNN approaches a tomographic image of the subject.

[0079] Next, a comparison will be made between a case where the sinogram is not divided into blocks and a case where the sinogram is divided into blocks with regard to the storage capacity required to store data in the RAM of the GPU.

[0080] Generally, a GPU is used in processing using CNN. A GPU is a calculation processing device specialized for image processing, and has a calculation unit and a RAM integrated on one semiconductor chip. Various data used in calculation processing by the calculation unit of the GPU is required to be stored in the RAM of the GPU. The data to be stored in the RAM of the GPU include, for example, a CNN input image, a CNN output image, a weighting coefficient representing the learning state of the CNN, a feature map, a measured sinogram, a calculated sinogram, parameters required for forward projection calculation, etc., and require a huge storage capacity. However, since the capacity of the RAM of the GPU is limited, in the above image processing method, although it is possible to perform a two-dimensional forward projection calculation, it may be difficult to perform a three-dimensional forward projection calculation.

[0081] Here, the number of pixels of the 3D output image created by the CNN is 128 × 128 × 64, and the number of pixels of the sinogram space is 128 × 128 × 64 × 19. In the image processing method when dividing the sinogram into blocks, K = 16 is set, and the 3D output image is forward projected to generate a calculated sinogram 24 equally divided into 16 blocks. 1 ~24 16 FIG. 6 shows a calculated sinogram 24 without block division and a calculated sinogram 24 with block division. 1 ~24 16 FIG. 6(a) is a diagram showing a comparison of the respective examples. FIG. 6(b) is a diagram showing a calculated sinogram 24 when block division is not performed. FIG. 6(c) is a diagram showing a calculated sinogram 24 when block division is performed. 1 ~24 16 is shown diagrammatically.

[0082] Calculation of each block when dividing into blocks Sinogram 24 k The number of pixels of the kth block is 128×8×64×19, which is 1 / 16 of the number of pixels of the calculated sinogram 24 when the block division is not performed. k Projection matrix P for forward projection calculation k The number of elements of the projection matrix P for performing forward projection calculation from the 3D output image to the calculated sinogram 24 when no block division is performed is 1 / 16 of the number of elements of the projection matrix P for performing forward projection calculation from the 3D output image to the calculated sinogram 24 when no block division is performed.

[0083] When dividing into blocks, the memory capacity required to store the data used in the forward projection calculation can be reduced compared to when dividing into blocks, and this data can be stored in the GPU RAM. Therefore, when dividing into blocks, 3D forward projection calculation from the CNN output image to the calculated sinogram becomes easy, and 3D tomographic images of the subject can be easily created by training the CNN based on the evaluation result of the error between the calculated sinogram and the measured sinogram.

[0084] Next, the evaluation function used by the CNN learning unit 15 in the CNN learning step will be further explained. In the explanation so far, the evaluation function used in the first processing step includes only an error evaluation term that represents an evaluation value regarding the error between the reconstructed image created by the image reconstruction unit 17 and the output image created by the convolution integral unit 13. In addition, the evaluation function used in the second processing step includes only an error evaluation term that represents an evaluation value regarding the error between the actual sinogram y 0 and the calculated sinogram y(=Pf(θ|z)).

[0085] However, in both the first and second processing steps, it is preferable to use an evaluation function that includes a regularization term in addition to the error evaluation term. The regularization term is for suppressing CNN overlearning and represents an evaluation value related to the difference in pixel values ​​between adjacent pixels in the output image.

[0086] That is, the evaluation function when the sinogram is not divided into blocks is given by the following equation (11) instead of the above equation (5). Also, the evaluation function when the sinogram is divided into blocks is given by the following equation (12) instead of the above equation (9). In these equations, the first term on the right side is an error evaluation term, and the second term on the right side is a regularization term. This regularization term imposes a penalty on the difference in pixel values ​​between adjacent pixels in the output image. β is a hyperparameter that adjusts the degree of the regularization effect. The smaller β is, the smaller the effect of regularization. The larger β is, the greater the effect of regularization (i.e., the effect of suppressing CNN overlearning).

[0087]

number

[0088]

number

[0089] The regularization term may represent an evaluation value regarding the difference in pixel values ​​between adjacent pixels in the output image 22 (f(θ|z)) output from the CNN processing unit 12, or may represent an evaluation value regarding the difference in pixel values ​​between adjacent pixels in the output image 23 (PSF(f(θ|z))) output from the convolution integral unit 13.

[0090] In the case of a two-dimensional image, the neighboring pixels of a given pixel include neighboring pixels in each of two mutually orthogonal directions, and preferably also include neighboring pixels in a diagonal direction. In the case of a two-dimensional image, the number of neighboring pixels of a given pixel is eight, excluding images located at an edge or corner of the image. In the case of a three-dimensional image, the neighboring pixels of a given pixel include neighboring pixels in each of three mutually orthogonal directions, and preferably also include neighboring pixels in a diagonal direction. In the case of a three-dimensional image, the number of neighboring pixels of a given pixel is 26, excluding images located at an edge or corner of the image.

[0091] FIG. 7 is a diagram for explaining adjacent pixels in an output image. This diagram shows the output image as a two-dimensional image, with 3×3 pixels among them. The pixel value of the pixel at the center of this diagram is λ j Let the pixel values ​​of the eight pixels adjacent to the central pixel be λ k (k=1 to 8), the difference in pixel value between adjacent pixels with respect to this central pixel is |λ j -λ k The regularization term represents an estimate of the difference between pixel values ​​for all combinations of adjacent pixels in the output image.

[0092] The regularization term may be expressed by various formulas as long as it represents an evaluation value related to the difference in pixel values ​​between adjacent pixels in the output image. For example, the regularization term is expressed by the following formula (13). In this formula (13), N j represents the set of pixels k adjacent to pixel j. γ is the pixel value λ jThis equation (13) contains a term for the difference in pixel values ​​of adjacent pixels in the numerator and a term for the sum of pixel values ​​of adjacent pixels in the denominator, and represents an evaluation value related to the relative difference in pixel values ​​between adjacent pixels in the output image.

[0093]

number

[0094] Incidentally, formula (13) is similar to the formula described in Non-Patent Document 2. However, in Non-Patent Document 1, the formula similar to formula (13) is used in the process of reconstructing a tomographic image of a subject based on coincidence counting information collected by a PET device, and is not used in the process of performing noise reduction processing on the tomographic image by the DIP technique.

[0095] In addition, as the regularization term, for example, Gibbs prior (Non-Patent Document 3) or total variation (Non-Patent Document 4) may be used. Note that these documents also describe a technique for reconstructing a tomographic image of a subject, but do not describe a technique for performing noise reduction processing on the tomographic image by the DIP technique.

[0096] Next, a description will be given of the results of creating simulation data by a Monte Carlo simulation of a head PET device using a digital brain phantom image, and using this to reconstruct a tomographic image by each of image processing methods 1 and 2. In image processing method 1 of the comparative example, a tomographic image was reconstructed by performing only the second processing step without performing the first processing step. In image processing method 2 of the embodiment, a tomographic image was reconstructed by performing the second processing step following the first processing step. In both image processing methods 1 and 2, in the second processing step, the actual sinogram and the calculated sinogram were each divided into multiple blocks, and a CNN was trained using an evaluation function that did not include a regularization term but only included an error evaluation term.

[0097] The phantom image used was a 3D brain image obtained from BrainWeb (https: / / brainweb.bic.mni.mcgill.ca / brainweb / ) with a simulated tumor embedded in the white matter. The number of pixels in the phantom image was 128 x 128 x 64. The number of pixels in the sinogram space was 128 x 128 x 64 x 19, and the sinogram space was equally divided into two blocks. The error evaluation term for the evaluation function was the mean squared error (MSE). The input image to the CNN was a 3D random noise image.

[0098] FIG. 8 is a diagram showing an original image (correct image) of a brain tomographic image. FIG. 9 is a diagram showing a brain tomographic image obtained by image processing method 1. FIG. 10 is a diagram showing a brain tomographic image obtained by image processing method 2. The PSNR of the tomographic image obtained by image processing method 1 (FIG. 9) is 20.70 dB, and the PSNR of the tomographic image obtained by image processing method 2 (FIG. 10) is 21.60 dB. PSNR (Peak Signal to Noise Ratio) represents the quality of an image in decibels (dB), and the higher the value, the better the image quality. Thus, compared to image processing method 1, image processing method 2 has a higher PSNR of the tomographic image and better noise reduction performance. Also, compared to image processing method 1, image processing method 2 can restore the structure of the gray matter in more detail.

[0099] In image processing method 1, the number of repetitions in the second processing step was 11, and the required time was about 165 minutes. In image processing method 2, the required time for the first processing step was several minutes, the number of repetitions in the second processing step was 3, and the required time for the second processing step was about 45 minutes. Thus, compared to image processing method 1, image processing method 2 required about one-third the time.

[0100] As described above, it has been confirmed that, compared with image processing method 1 which performs only the second processing step, image processing method 2 which performs the first processing step followed by the second processing step can obtain cross-sectional images with reduced noise in a shorter time.

[0101] In this embodiment, after CNN is trained to a certain extent in a short time in the first processing step, CNN is further trained in the subsequent second processing step, thereby making it possible to shorten the overall processing time including the time required for the second processing step, and to obtain a tomographic image with reduced noise. [Explanation of symbols]

[0102] 1...radiation tomography system, 2...radiation tomography device, 10...image processing device, 11...sinogram creation unit, 12...CNN processing unit, 13...convolution integral unit, 14...forward projection calculation unit, 15...CNN learning unit, 16...memory unit, 17...image reconstruction unit.

Claims

1. 1. An image processing device that creates a tomographic image of a subject based on coincidence counting information collected by a radiation tomography device having a plurality of detectors arranged around a measurement space in which a subject to which an RI radiation source is administered is placed, comprising: an image reconstruction unit that creates a reconstructed image by an analytical image reconstruction method or a statistical image reconstruction method based on the coincidence count information collected by the radiation tomography apparatus; a sinogram creating unit that creates a sinogram based on coincidence information collected by the radiation tomography apparatus; A CNN processing unit that inputs an input image to a convolutional neural network and creates an output image by the convolutional neural network; a forward projection calculation unit for calculating a forward projection of the output image to create a sinogram; A CNN learning unit that trains the convolutional neural network; Equipped with In the first processing step, the CNN learning unit uses an evaluation function including an error evaluation term that represents an evaluation value regarding an error between the reconstructed image and the output image, and trains the convolutional neural network based on a value of the evaluation function; The processing of the CNN processing unit and the CNN learning unit are repeated a plurality of times; In a second processing step following the first processing step, the CNN learning unit uses an evaluation function including an error evaluation term that represents an evaluation value regarding an error between the sinogram created by the sinogram creation unit and the sinogram created by the forward projection calculation unit, and causes the convolutional neural network to learn based on the value of this evaluation function; The output image obtained by repeatedly performing the processes of the CNN processing unit, the forward projection calculation unit, and the CNN learning unit a plurality of times is set as a tomographic image of the subject. Image processing device.

2. the sinogram creation unit creates a sinogram divided into a plurality of blocks based on coincidence information collected by the radiation tomography apparatus; the forward projection calculation unit performs a forward projection calculation on the output image to generate a sinogram divided into the plurality of blocks; In the second processing step, the CNN learning unit trains the convolutional neural network based on the value of the evaluation function for each of the plurality of blocks. The image processing device according to claim 1 .

3. each of the tomographic image, the reconstructed image, the input image, and the output image is a three-dimensional image; The image processing device according to claim 1 .

4. a convolution integral unit that performs a convolution integral of a point spread function on the output image, The forward projection calculation unit performs forward projection calculation on the output image after processing by the convolution integral unit. The image processing device according to claim 1 .

5. In the second processing step, the CNN learning unit evaluates the error by the error evaluation term in a region in a sinogram space where coincidence information can be collected by the radiation tomography apparatus. The image processing device according to claim 1 .

6. The CNN learning unit uses an evaluation function further including a regularization term that represents an evaluation value related to a difference in pixel values ​​between adjacent pixels in the output image, and trains the convolutional neural network based on the value of the evaluation function. The image processing device according to claim 1 .

7. The CNN processing unit inputs an image representing morphological information of the subject as the input image to the convolutional neural network. The image processing device according to claim 1 .

8. The CNN processing unit inputs an MRI image of the subject as the input image to the convolutional neural network. The image processing device according to claim 1 .

9. The CNN processing unit inputs a CT image of the subject as the input image to the convolutional neural network. The image processing device according to claim 1 .

10. The CNN processing unit inputs a static PET image of the subject as the input image to the convolutional neural network. The image processing device according to claim 1 .

11. The CNN processing unit inputs a random noise image as the input image to the convolutional neural network. The image processing device according to claim 1 .

12. a radiation tomography apparatus having a plurality of detectors arranged around a measurement space in which a subject to which an RI radiation source is administered is placed, and for collecting coincidence count information; An image processing device according to any one of claims 1 to 11, which creates a tomographic image of the subject based on coincidence count information collected by the radiation tomography device; A radiation tomography system comprising:

13. 1. An image processing method for creating a tomographic image of a subject based on coincidence counting information collected by a radiation tomography apparatus having a plurality of detectors arranged around a measurement space in which a subject to which an RI radiation source is administered is placed, comprising: an image reconstruction step of generating a reconstructed image by an analytical image reconstruction method or a statistical image reconstruction method based on the coincidence count information collected by the radiation tomography apparatus; a sinogram creation step of creating a sinogram based on the coincidence count information collected by the radiation tomography apparatus; A CNN processing step of inputting an input image into a convolutional neural network and creating an output image by the convolutional neural network; a forward projection calculation step of calculating a forward projection of the output image to generate a sinogram; a CNN learning step of training the convolutional neural network; Equipped with In the first processing step, the CNN learning step uses an evaluation function including an error evaluation term that represents an evaluation value regarding an error between the reconstructed image and the output image, and learns the convolutional neural network based on the value of the evaluation function; The CNN processing step and the CNN learning step are each repeated a plurality of times; In a second processing step following the first processing step, the CNN learning step uses an evaluation function including an error evaluation term that represents an evaluation value regarding an error between the sinogram created by the sinogram creation step and the sinogram created by the forward projection calculation step, and causes the convolutional neural network to learn based on the value of this evaluation function; The output image obtained by repeatedly performing the CNN processing step, the forward projection calculation step, and the CNN learning step a plurality of times is set as a tomographic image of the subject. Image processing methods.

14. The sinogram creation step creates a sinogram divided into a plurality of blocks based on coincidence information collected by the radiation tomography apparatus, The forward projection calculation step performs a forward projection calculation on the output image to generate a sinogram divided into the plurality of blocks; In the second processing step, the CNN learning step trains the convolutional neural network based on the value of the evaluation function for each of the plurality of blocks. The image processing method according to claim 13.

15. each of the tomographic image, the reconstructed image, the input image, and the output image is a three-dimensional image; The image processing method according to claim 13.

16. A convolution integral step of performing a convolution integral of a point spread function on the output image, The forward projection calculation step performs forward projection calculation on an output image after processing by the convolution integral step. The image processing method according to claim 13.

17. In the second processing step, the CNN learning step evaluates the error by the error evaluation term in a region in a sinogram space where coincidence information can be collected by the radiation tomography apparatus. The image processing method according to claim 13.

18. The CNN learning step uses an evaluation function further including a regularization term representing an evaluation value related to a difference in pixel values ​​between adjacent pixels in the output image, and learns the convolutional neural network based on the value of the evaluation function. The image processing method according to claim 13.

19. The CNN processing step inputs an image representing morphological information of the subject as the input image to the convolutional neural network. The image processing method according to claim 13.

20. The CNN processing step inputs an MRI image of the subject as the input image to the convolutional neural network. The image processing method according to claim 13.

21. The CNN processing step inputs a CT image of the subject as the input image to the convolutional neural network. The image processing method according to claim 13.

22. The CNN processing step inputs a static PET image of the subject as the input image to the convolutional neural network. The image processing method according to claim 13.

23. The CNN processing step inputs a random noise image as the input image to the convolutional neural network; The image processing method according to claim 13.