Image processing apparatus and image processing method

Abstract

To provide an image processing device with which, when creating the tomographic image of an analyte after training a CNN on the basis of the evaluation result of error between a computational sinogram and a measured sinogram, it is possible to suppress the degradation of image quality due to CNN over-training in a noise reduction process using DIP technology.SOLUTION: An image processing device 10 comprises a sinogram creation unit 11, a CNN processing unit 12, a convolutional integration unit 13, a forward projection calculation unit 14, and a CNN training unit 15. The forward projection calculation unit 14 performs a forward projection calculation on an output image 23 and creates a computational sinogram 24. The CNN training unit 15 trains the CNN using the value of an evaluation function that includes an error evaluation term that represents an evaluation value regarding an error between a measured sinogram 21 and the computational sinogram 24 and a regularization term that represents an evaluation value regarding a difference in pixel value between adjacent pixels in output images.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an apparatus and method for creating a tomographic image of a subject based on simultaneous counting information collected by a tomography apparatus. [Background technology]

[0002] Examples of radiographic tomography devices that can acquire cross-sectional images of a subject (living organism) include PET (Positron Emission Tomography) devices and SPECT (Single Photon Emission Computed Tomography) devices.

[0003] A PET scanner has a detection unit equipped with numerous small radiation detectors arranged around the measurement space where the subject is placed. The PET scanner detects photon pairs with an energy of 511 keV generated by electron-positron annihilation in a subject administered with a positron-emitting isotope (RI source) using the coincidence counting method, and collects this coincidence counting information. Based on this collected information, a tomographic image representing the spatial distribution of the frequency of photon pair generation in the measurement space (i.e., the spatial distribution of the RI source) can be reconstructed. This PET scanner plays an important role in fields such as nuclear medicine, and can be used to study, for example, biological functions and higher brain functions.

[0004] Various methods are known for reconstructing tomographic images of a subject based on a large amount of collected simultaneous counting information. The image processing method for tomographic image reconstruction described in Non-Patent Literature 1 reconstructs tomographic images using Deep Image Prior technology, which utilizes a convolutional neural network, a type of deep neural network. Hereafter, Convolutional Neural Network will be referred to as "CNN," and Deep Image Prior technology will be referred to as "DIP technology." DIP technology utilizes the property of CNNs that meaningful structures in an image are learned faster than random noise (i.e., random noise is difficult to learn). DIP technology makes it possible to obtain tomographic images with reduced noise.

[0005] The image processing method described in Non-Patent Document 1 is specifically as follows: A sinogram (hereinafter referred to as "measured sinogram") is created based on a large amount of simultaneous counting information collected for the subject. In addition, when an input image (e.g., an MRI image) is input to a CNN, the image output from the CNN is subjected to forward projection calculation (Radon transform) to create a sinogram (hereinafter referred to as "computed sinogram"). The error between this computed sinogram and the measured sinogram is then evaluated, and the CNN is trained based on this error evaluation result. By repeating the image output from the CNN, the creation of a computed sinogram by forward projection calculation, the error evaluation, and the training of the CNN using DIP technology, the computed sinogram gradually approaches the measured sinogram, and the output image from the CNN approaches the tomographic image of the subject.

[0006] This image processing method includes forward projection from the CNN output image to the computed sinogram, but does not include back projection from the measured sinogram to the tomographic image. Therefore, it is possible to obtain tomographic images with reduced noise.

[0007] A sinogram is a histogram representing the frequency of simultaneous counting information acquisition (frequency of simultaneous counting events) in a space (sinogram space) represented by four variables r, θ, z, and δ. The variable r represents the distance from the central axis to the simultaneous counting line (the line connecting two detectors that simultaneously counted photon pairs). The variable θ represents the azimuth angle of the simultaneous counting line. The variable z represents the position of the midpoint of the simultaneous counting line along the central axis. The variable δ represents the distance along the central axis between two detectors that simultaneously counted photon pairs. [Prior art documents] [Non-patent literature]

[0008] [Non-Patent Document 1] F. Hashimoto, K. Ote and Y. Onishi, "PET Image Reconstruction Incorporating Deep Image Prior and aForward Projection Model," IEEE Transactions on Radiation and PlasmaMedical Sciences, doi: 10.1109 / TRPMS.2022.3161569. [Non-Patent Document 2] J. Nuytset al. "A concave prior penalizing relative differences for maximum-a-posteriori reconstruction in emission tomography." IEEE TNS.49:1, 56-60, 2002. [Non-Patent Document 3] Hiroshi Kudo, "Image Reconstruction Methods in Low-Radiation CT - Fundamentals of Statistical Image Reconstruction, Iterative Reconstruction, and Compressed Sensing," Medical Imaging Technology 32.4 (2014): 239-248. [Non-Patent Document 4] Chambolle, Antonin. "An algorithm for total variation minimization and applications." Journal of Mathematical imaging and vision 20.1 (2004): 89-97.

Summary of the Invention

Problems to be Solved by the Invention

[0009] Although the noise reduction process using DIP technology has excellent noise reduction performance, it has the problem of image quality degradation due to overfitting of CNN. That is, as described above, the DIP technology utilizes the property of CNN that random noise is difficult to be learned. However, as the number of CNN learning increases, random noise will also be restored. Thus, due to the overfitting of CNN, random noise is restored, resulting in image quality degradation.

[0010] The present invention is made to solve the above problems. When creating a tomographic image of a subject by training a CNN based on the evaluation result of the error between a calculated sinogram and a measured sinogram, the present invention aims to provide an image processing apparatus and an image processing method capable of suppressing image quality degradation due to CNN overfitting in the noise reduction process using DIP technology and obtaining a tomographic image with reduced noise.

Means for Solving the Problems

[0011] The image processing apparatus of the present invention is an image processing apparatus that creates a tomographic image of a subject based on simultaneous counting information collected by a radiation tomography apparatus having a plurality of detectors arranged to surround a measurement space in which the subject administered with RI rays is placed.

[0012] The first aspect of the image processing apparatus of the present invention includes: (1) a sinogram creation unit that creates a sinogram based on coincidence count information collected by a radiation tomography apparatus; (2) a CNN processing unit that inputs an input image into a convolutional neural network and creates an output image by the convolutional neural network; (3) a forward projection calculation unit that performs a forward projection calculation on the output image to create a sinogram; and (4) an error evaluation term representing an evaluation value regarding the error between the sinogram created by the sinogram creation unit and the sinogram created by the forward projection calculation unit, and a regularization term representing an evaluation value regarding the difference in pixel values between adjacent pixels in the output image. A CNN learning unit that uses an evaluation function including these and learns the convolutional neural network based on the value of this evaluation function. Then, the output image after repeatedly performing the processes of the CNN processing unit, the forward projection calculation unit, and the CNN learning unit a plurality of times is used as a tomographic image of the subject.

[0013] The image processing apparatus of the present invention may also be in the following aspects. In the second aspect, in addition to the first aspect, the sinogram creation unit creates a sinogram divided into a plurality of blocks based on the coincidence count information collected by the radiation tomography apparatus, the forward projection calculation unit performs a forward projection calculation on the output image to create a sinogram divided into a plurality of blocks, and the CNN learning unit learns the convolutional neural network based on the value of the evaluation function for each of the plurality of blocks.

[0014] In the third aspect, in addition to the first aspect or the second aspect, each of the tomographic image, the input image, and the output image is a three-dimensional image.

[0015] In the fourth aspect, in addition to any one of the first to third aspects, the image processing apparatus further includes a convolution integration unit that performs a convolution integration of a point spread function on the output image, and the forward projection calculation unit performs a forward projection calculation on the output image after the processing by the convolution integration unit.

[0016] In the fifth embodiment, in addition to any of the first to fourth embodiments, the CNN learning unit evaluates the error using an error evaluation term in a region of the sinogram space where simultaneous counting information can be collected by a radiographic tomography device.

[0017] In the sixth embodiment, in addition to any of the first to fifth embodiments, the CNN processing unit inputs an image representing the morphological information of the subject, an MRI image of the subject, a CT image of the subject, a static PET image of the subject, or a random noise image as input images to the convolutional neural network.

[0018] The radiographic tomography system of the present invention comprises a radiographic tomography apparatus having a plurality of detectors arranged around a measurement space in which a subject administered with an RI source is placed and which collects coincidence count information, and the above-described image processing apparatus of the present invention which creates a tomographic image of the subject based on the coincidence count information collected by the radiographic tomography apparatus.

[0019] The present invention is an image processing method that creates a tomographic image of a subject based on simultaneous counting information collected by a tomography apparatus having multiple detectors arranged around a measurement space in which a subject administered with an RI source is placed.

[0020] A first aspect of the image processing method of the present invention comprises: (1) a sinogram creation step of creating a sinogram based on coincidence counting information collected by a tomography device; (2) a CNN processing step of inputting an input image to a convolutional neural network and creating an output image using the convolutional neural network; (3) a forward projection calculation step of creating a sinogram by performing forward projection calculation on the output image; and (4) a CNN learning step of training a convolutional neural network based on the value of an evaluation function that includes an error evaluation term representing an evaluation value regarding the error between the sinogram created in the sinogram creation step and the sinogram created in the forward projection calculation step, and a regularization term representing an evaluation value regarding the difference in pixel values ​​between adjacent pixels in the output image. The output image after repeating each of the CNN processing step, forward projection calculation step and CNN learning step multiple times is used as the tomographic image of the subject.

[0021] The image processing method of the present invention may also be in the following embodiments. In the second embodiment, in addition to the first embodiment, in the sinogram creation step, a sinogram divided into multiple blocks is created based on the simultaneous counting information collected by the tomography device; in the forward projection calculation step, the output image is forward projected to create a sinogram divided into multiple blocks; and in the CNN learning step, a convolutional neural network is trained based on the evaluation function values ​​for each of the multiple blocks.

[0022] In the third embodiment, in addition to the first or second embodiment, the tomographic image, input image, and output image are all three-dimensional images.

[0023] In the fourth embodiment, in addition to any of the first to third embodiments, the image processing method further comprises a convolution integral step in which a convolution integral of the point image distribution function is performed on the output image, and in the forward projection calculation step, the output image after processing by the convolution integral step is subjected to forward projection calculation.

[0024] In the fifth embodiment, in addition to any of the first to fourth embodiments, the error is evaluated by an error evaluation term in the CNN learning step in a region of the sinogram space where simultaneous counting information can be collected using a radiographic tomography device.

[0025] In the sixth embodiment, in addition to any of the first to fifth embodiments, in the CNN processing step, an image representing the morphological information of the subject, an MRI image of the subject, a CT image of the subject, a static PET image of the subject, or a random noise image is input to the convolutional neural network as an input image. [Effects of the Invention]

[0026] According to the present invention, when creating a tomographic image of a subject by training a CNN based on the evaluation result of the error between a computed sinogram and a measured sinogram, it is possible to suppress image quality degradation due to CNN overfitting in the noise reduction process using DIP technology and obtain a tomographic image with reduced noise. [Brief explanation of the drawing]

[0027] [Figure 1] Figure 1 shows the configuration of the radiographic tomography system 1. [Figure 2] Figure 2 shows an example of a CNN configuration. [Figure 3] Figure 3 is a flowchart of the image processing method. [Figure 4] Figure 4 shows a comparison of calculation sinograms 24 without block division and calculation sinograms 241 to 2416 with block division. Figure 4(a) schematically shows calculation sinogram 24 without block division. Figure 4(b) schematically shows calculation sinograms 241 to 2416 with block division. [Figure 5] Figure 5 illustrates adjacent pixels in the output image. [Figure 6] Figure 6 shows cross-sectional images of the brain obtained using image processing method 1. [Figure 7]Figure 7 shows cross-sectional images of the brain obtained using image processing method 2. [Figure 8] Figure 8 shows cross-sectional images of the brain obtained using image processing method 3. [Modes for carrying out the invention]

[0028] Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the attached drawings. In the description of the drawings, the same elements will be denoted by the same reference numerals, and redundant descriptions will be omitted. The present invention is not limited to these examples, but is indicated by the claims, and all modifications within the meaning and scope equivalent to the claims are intended to be included.

[0029] Figure 1 shows the configuration of the radiometric tomography system 1. The radiometric tomography system 1 comprises a radiometric tomography device 2 and an image processing device 10. The image processing device 10 comprises a sinogram creation unit 11, a CNN processing unit 12, a convolution integration unit 13, a forward projection calculation unit 14, and a CNN learning unit 15.

[0030] The input image, output image, and tomographic image may be either two-dimensional or three-dimensional, but in the following explanation, we will assume that these images are three-dimensional. Furthermore, the measured sinogram and calculated sinogram may or may not be divided into multiple blocks, but in the following explanation, we will mainly focus on the case where these sinograms are divided into multiple blocks.

[0031] The tomography system 2 is a device that collects coincidence count information for reconstructing tomographic images of a subject. Examples of tomography systems 2 include PET scanners and SPECT scanners. In the following explanation, we will assume that the tomography system 2 is a PET scanner.

[0032] The tomography apparatus 2 is equipped with a detection unit having a number of small radiation detectors arranged around the measurement space where the subject is placed. The tomography apparatus 2 detects photon pairs with an energy of 511 keV generated by electron-positron annihilation in the subject to which the RI source has been administered, using the coincidence counting method, and collects this coincidence counting information. The tomography apparatus 2 then outputs this collected coincidence counting information to the image processing device 10.

[0033] The image processing device 10 includes a GPU (Graphics Processing Unit) that performs processing using a convolutional neural network (CNN), an input unit that accepts user input (e.g., a keyboard or mouse), a display unit that displays images (e.g., a liquid crystal display), and a storage unit that stores programs and data for executing various processes. A computer having a CPU, RAM, ROM, and a hard disk drive is used as the image processing device 10.

[0034] The sinogram creation unit 11 creates a measured sinogram 21 based on the synchronization counting information collected by the tomography device 2. At this time, the sinogram creation unit 11 divides the measured sinogram 211 to 21 into multiple (K) blocks. K Create the actual sinogram 21. k This is the measured sinogram of the k-th block out of K blocks. K is an integer greater than or equal to 2, and k is an integer between 1 and K (inclusive). Divided measured sinograms 211-21 K The combined result is the overall measured sinogram 21.

[0035] The CNN processing unit 12 inputs a 3D input image 20 to the CNN and uses the CNN to create a 3D output image 22. The 3D input image 20 may be an image representing the morphological information of a subject, an MRI image, CT image, or static PET image of the subject, or a random noise image.

[0036] The convolution integration unit 13 performs convolution integration of the point spread function on the 3D output image 22 created by the CNN processing unit 12 to create a new 3D output image 23. The point spread function (PSF) is a function representing the response (impulse response) of the radiation tomography apparatus to a dotted line source, and is generally represented by a Gaussian function, or an asymmetric Gaussian function with different blurring depending on the position within the field of view modeled from the measured data of the dotted line source. By providing the convolution integration unit 13, a tomographic image with better image quality can be obtained, and the learning of the CNN can be stabilized.

[0037] The forward projection calculation unit 14 performs forward projection calculation on the 3D output image 23 to create a calculated sinogram 24. At this time, the forward projection calculation unit 14 creates calculated sinograms 241 to 24 K which are created. The calculated sinogram 24 k is the calculated sinogram of the k-th block among the K blocks. The combined calculated sinograms 241 to 24 K are the overall calculated sinogram 24.

[0038] The block division of the calculated sinogram 24 is performed in the same manner as the block division of the measured sinogram 21. The calculated sinogram 24 k of the k-th block and the measured sinogram 21 k of the k-th block are sinograms of a common region in the overall sinogram space. The mode of block division is arbitrary, and block division may be performed on any one or two or more of the four variables representing the sinogram space. The sizes of each of the K blocks may be different or the same.

[0039] The CNN learning unit 15 evaluates the error between the measured sinogram 21 k and the calculated sinogram 24 k for each of the K blocks, and learns the CNN based on the error evaluation results for each of the K blocks.

[0040] The 3D output image 22 created by the CNN processing unit 12 after the CNN processing unit 12, convolution integration unit 13, forward projection calculation unit 14, and CNN learning unit 15 have been repeated multiple times is used as the 3D tomographic image of the subject. Alternatively, the 3D output image 23 created by the convolution integration unit 13 may be used as the 3D tomographic image of the subject. Since the measured sinogram 21 reflects the response function of the radiographic tomography device, it is preferable to use the 3D output image 22 before the convolution integration of the point image distribution function by the convolution integration unit 13 as the 3D tomographic image of the subject.

[0041] The convolution integral unit 13 may be provided as the final layer of the CNN, or it may be provided separately from the CNN. If the convolution integral unit 13 is provided as the final layer of the CNN, the weight coefficients of the convolution integral unit 13 are kept constant during CNN training. Alternatively, the convolution integral unit 13 may not be provided. If the convolution integral unit 13 is not provided, the forward projection calculation unit 14 performs a forward projection calculation on the 3D output image 22 output from the CNN processing unit 12 to create a computational sinogram 24.

[0042] Figure 2 shows an example of a CNN configuration. The CNN shown in this figure has a 3D U-net structure that includes an encoder and a decoder. In this figure, the size of each layer of the CNN is shown, with the number of pixels in the 3D input image 20 input to the CNN being N×N×64.

[0043] Figure 3 is a flowchart of the image processing method. The image processing method comprises a sinogram creation step S1 performed by a sinogram creation unit 11, a CNN processing step S2 performed by a CNN processing unit 12, a convolution integration step S3 performed by a convolution integration unit 13, a forward projection calculation step S4 performed by a forward projection calculation unit 14, and a CNN learning step S5 performed by a CNN learning unit 15.

[0044] In the sinogram creation step S1, based on the synchronization counting information collected by the tomography device 2, the measured sinograms 211-21 are divided into K blocks. K The CNN is created. In the CNN processing step S2, the CNN is input with the 3D input image 20, and the CNN creates the 3D output image 22. In the convolution integral step S3, the CNN is subjected to a convolution integral of the point image distribution function on the 3D output image 22 created in the CNN processing step S2 to create a new 3D output image 23.

[0045] In the forward projection calculation step S4, the 3D output image 23 is subjected to forward projection calculation and divided into K blocks, resulting in the computational sinograms 241-24. K Create the following: In CNN training step S5, for each of the K blocks, the measured sinogram 21 k and calculation sinogram 24 k The error between the given values ​​is evaluated, and the CNN is trained based on the error evaluation results for each of the K blocks.

[0046] After repeating the CNN processing step S2, the convolution and integration step S3, the forward projection calculation step S4, and the CNN learning step S5 multiple times, the 3D output image 22 created in the CNN processing step S2 is used as the 3D tomographic image of the subject. Alternatively, the 3D output image 23 created in the convolution and integration step S3 may be used as the 3D tomographic image of the subject. Note that the convolution and integration step S3 is optional.

[0047] Next, we will explain the processing steps of the image processing method when the sinogram is not divided into multiple blocks. In the image processing method when the sinogram is not divided into blocks, the entire measured sinogram and the calculated sinogram are processed.

[0048] In the following, let f be the CNN processing, z be the 3D input image 20 input to the CNN, and θ be the weight coefficient parameter representing the CNN's learning state. θ changes as the CNN's learning progresses. Let x be the 3D output image 22 output from the CNN when the 3D input image z is input to a CNN with weight coefficients of θ. The 3D output image x is expressed by (1) below. In the CNN processing step, the 3D output image x is created by performing the processing represented by this equation.

[0049]

number

[0050] In the convolution integral step, a new 3D output image x is created by performing a convolution integral of the point image distribution function on the 3D output image x created in the CNN processing step. In Figure 1, the 3D output image x after the convolution integral is denoted as PSF(f(θ|z)).

[0051] In the forward projection calculation step, the 3D output image x is forward projected to create a computed sinogram 24. Let the computed sinogram 24 be y, and let P be the projection matrix used to perform the forward projection calculation (Radon transform) from the 3D output image x to the computed sinogram y. The projection matrix is ​​also called the system matrix or detection probability matrix. The process performed in the forward projection calculation step is expressed by equation (2) below.

[0052]

number

[0053] In the CNN training step, the measured sinogram 21 is taken as y0, and the error between the measured sinogram y0 and the calculated sinogram y (equation (2) above) is evaluated, and the CNN is trained based on the result of this error evaluation. The processing performed in the CNN training step is expressed by equation (3) below. The constrained optimization problem in this equation is the problem of optimizing the CNN parameters θ so that the value of the evaluation function E(y;y0) is small, under the constraint that the 3D output image x created by the CNN is a tomographic image of the subject.

[0054]

number

[0055] The constrained optimization problem in equation (3) can be transformed into the unconstrained optimization problem in equation (4) below. The evaluation function E can be arbitrary, but for example, the L1 norm, the L2 norm, or the negative log-likelihood in the Poisson distribution can be used. If the L2 norm is used as the evaluation function, equation (4) can be transformed into equation (5) below.

[0056]

number

[0057]

number

[0058] Considering the arrangement of multiple detectors in a tomography system, there may be regions in the sinogram space where it is impossible to collect coincidence information. For this reason, the optimization problem in equation (5) above may be replaced with the optimization problem in equation (6) below. In equation (6), m is a binary mask function, which has a value of 1 in regions of the sinogram space where it is possible to collect coincidence information, and a value of 0 in regions where it is impossible. Equation (6) selectively evaluates the error in regions of the sinogram space where it is possible to collect coincidence information by taking the Hadamard product of the error (y-y0) and the binary mask function m.

[0059]

number

[0060] By repeatedly performing each of the CNN processing steps—the convolution integration step, the forward projection calculation step, and the CNN learning step—and solving this optimization problem for the CNN parameter θ, the computed sinogram y approaches the measured sinogram y0, and the 3D output image x created by the CNN approaches the tomographic image of the subject.

[0061] Next, we will explain in detail the processing steps of the image processing method when dividing a sinogram into blocks. When dividing a sinogram into blocks, in the forward projection calculation step, the 3D output image x is forward projected and divided into K blocks, resulting in the calculated sinogram 241-24 K Create the kth block's computational sinogram 24. k to y k Then, the sinogram y is calculated from the 3D output image x. k The projection matrix P is used to perform a forward projection calculation (Radon transform) onto the matrix. k The process performed in the forward projection calculation step is expressed by equation (7) below.

[0062]

number

[0063] In the CNN training step, the measured sinogram 21 of block k k to y 0k For each of the K blocks, the measured sinogram y 0k and calculate sinogram y k The error between and is evaluated, and the CNN is trained based on the result of this error evaluation for each of the K blocks. The process performed in the CNN training step is represented by the unconstrained optimization problem in equation (8) below. If the L2 norm is used as the evaluation function, equation (8) can be transformed into equation (9) below. Furthermore, if the error is selectively evaluated in a region of the sinogram space where coincidence information can be collected, it can be represented by the unconstrained optimization problem in equation (10) below. m k This is the binary mask function in the k-th block.

[0064]

number

[0065]

number

[0066]

number

[0067] By repeatedly performing the CNN processing step, convolution integration step, forward projection calculation step, and CNN learning step multiple times, and solving this optimization problem for the CNN parameter θ, a computed sinogram y is obtained for each of the K blocks. k This is the measured sinogram y 0k As it approaches the subject, the 3D output image x created by the CNN approaches the tomographic image of the subject.

[0068] Next, we will discuss the memory capacity required to store data in the GPU's RAM, comparing the case where the sinogram is not block-divided and the case where it is block-divided.

[0069] Generally, GPUs are used in processing with CNNs. A GPU is a computing device specialized for image processing, and it has an integrated computing unit and RAM on a single semiconductor chip. Various types of data used in the computing processing performed by the GPU's computing unit are required to be stored in the GPU's RAM. The data that should be stored in the GPU's RAM includes, for example, CNN input images, CNN output images, weight coefficients representing the CNN's learning state, feature maps, measured sinograms, computed sinograms, and parameters necessary for forward projection calculations, requiring a huge amount of storage capacity. However, because the capacity of GPU RAM is limited, while it is possible to perform 2D forward projection calculations with the image processing method described above, it can be difficult to perform 3D forward projection calculations.

[0070] Here, the number of pixels in the 3D output image created by the CNN is set to 128 × 128 × 64, and the number of pixels in the sinogram space is set to 128 × 128 × 64 × 19. In the image processing method when dividing the sinogram into blocks, K=16 is used, and the 3D output image is forward-projected to divide it into 16 equally divided blocks, resulting in the calculated sinograms 241-24. 16 The following will be created. Figure 4 shows the calculation sinogram 24 without block division and the calculation sinograms 241-24 with block division. 16 This figure shows a comparison of each example. Figure 4(a) schematically shows the computational sinogram 24 when block division is not performed. Figure 4(b) shows the computational sinograms 241-24 when block division is performed. 16 This is illustrated schematically.

[0071] Calculation sinogram 24 for each block when dividing into blocks kThe number of pixels is 128 × 8 × 64 × 19, which is 1 / 16 of the number of pixels in the calculated sinogram 24 when block division is not performed. Also, when block division is performed, the calculated sinogram 24 of the kth block is obtained from the 3D output image. k Projection matrix P for performing forward projection calculations. k The number of elements is 1 / 16 of the number of elements in the projection matrix P used to perform forward projection calculations from the 3D output image to the computed sinogram 24 when block division is not performed.

[0072] When block partitioning is used, the amount of memory required to store the data used in forward projection calculations can be reduced compared to when block partitioning is not used, and this data can be stored in the GPU's RAM. Therefore, when block partitioning is used, 3D forward projection calculations from CNN output images to computed sinograms become easier, and 3D tomographic images of the subject can be easily created by training the CNN based on the evaluation results of the error between the computed sinogram and the measured sinogram.

[0073] Next, we will further explain the evaluation function used by the CNN learning unit 15 in CNN learning step S5. The evaluation functions described so far (equations (5) and (9)) only included an error evaluation term that represents the evaluation value of the error between the measured sinogram y0 and the calculated sinogram y (=Pf(θ|z)). However, it is preferable to use an evaluation function that also includes a regularization term in addition to this error evaluation term. The regularization term is used to suppress CNN overfitting and represents the evaluation value of the difference in pixel values ​​between adjacent pixels in the output image.

[0074] In other words, the evaluation function when the sinogram is not divided into blocks is given by equation (11) below, instead of equation (5) above. Also, the evaluation function when the sinogram is divided into blocks is given by equation (12) below, instead of equation (9) above. In these equations, the first term on the right-hand side is the error evaluation term, and the second term on the right-hand side is the regularization term. This regularization term imposes a penalty on the difference in pixel values ​​between adjacent pixels in the output image. β is a hyperparameter that adjusts the degree of the regularization effect. The smaller β is, the smaller the regularization effect. The larger β is, the greater the regularization effect (i.e., the effect of suppressing CNN overfitting).

[0075]

number

[0076]

number

[0077] The regularization term may represent an evaluation value relating to the difference in pixel values ​​between adjacent pixels in the output image 22(f(θ|z)) output from the CNN processing unit 12, or it may represent an evaluation value relating to the difference in pixel values ​​between adjacent pixels in the output image 23(PSF(f(θ|z)))) output from the convolution integral unit 13.

[0078] In a two-dimensional image, the pixels adjacent to a given pixel include pixels adjacent in two mutually orthogonal directions, and preferably also include pixels adjacent in diagonal directions. In a two-dimensional image, the number of pixels adjacent to a given pixel is 8, excluding pixels located at the edges or corners of the image. In a three-dimensional image, the pixels adjacent to a given pixel include pixels adjacent in three mutually orthogonal directions, and preferably also include pixels adjacent in diagonal directions. In a three-dimensional image, the number of pixels adjacent to a given pixel is 26, excluding pixels located at the edges or corners of the image.

[0079] Figure 5 illustrates adjacent pixels in the output image. This figure shows the output image as a 2D image, specifically a 3x3 pixel area. The pixel value of the central pixel in this figure is λ. j The pixel values ​​of the eight pixels adjacent to this central pixel are λ k If we consider (k=1~8), the difference in pixel values ​​between adjacent pixels with respect to this central pixel is |λ j -λ k It is represented by |. The regularization term represents an evaluation value for the difference in pixel values ​​for all combinations of adjacent pixels in the output image.

[0080] The regularization term can represent an evaluation value related to the difference in pixel values ​​between adjacent pixels in the output image, and can be expressed by various formulas. For example, the regularization term is expressed by the following formula (13). In this formula (13), N j γ represents the set of pixels k adjacent to pixel j. γ is the pixel value λ. j This represents the magnitude of the change in the value of the regularization term in response to a change in . Equation (13) includes a term representing the difference in pixel values ​​of adjacent pixels in the numerator and a term representing the sum of pixel values ​​of adjacent pixels in the denominator, and represents an evaluation value regarding the relative difference in pixel values ​​between adjacent pixels in the output image.

[0081]

number

[0082] Furthermore, equation (13) is similar to the equation described in Non-Patent Document 2. However, in Non-Patent Document 1, the equation similar to equation (13) is used in the process of reconstructing the tomographic image of the subject based on the simultaneous counting information collected by the PET device, and not in the process of noise reduction processing of the tomographic image using DIP technology.

[0083] Furthermore, as a regularization term, for example, Gibbs prior (Non-Patent Literature 3) or Total variation (Non-Patent Literature 4) may be used. Note that these documents describe techniques for reconstructing tomographic images of a subject, and do not describe techniques for noise reduction processing of tomographic images using DIP (Digital Injection Processing) techniques.

[0084] Next, simulation data was created using a Monte Carlo simulation of a head PET scanner with digital brain phantom images, and the results of reconstructing tomographic images using each of the image processing methods 1 to 3 will be explained. In image processing method 1, tomographic images were reconstructed using the ML-EM (Maximum Likelihood Expectation Maximization) method, which is a common image reconstruction method. In image processing method 2, tomographic images were reconstructed using the evaluation function of equation (9) above in the image processing method described using Figures 1 to 4. In image processing method 3, tomographic images were reconstructed using the evaluation functions of equations (12) and (13) above in the image processing method described using Figures 1 to 4.

[0085] As phantom images, we used 3D images obtained from BrainWeb (https: / / brainweb.bic.mni.mcgill.ca / brainweb / ) with simulated tumors embedded in the white matter. The phantom image had 128 × 128 × 64 pixels. In image processing methods 2 and 3, the sinogram space had 128 × 128 × 64 × 19 pixels, and the sinogram space was divided equally into two blocks. The error evaluation term of the evaluation function used in image processing methods 2 and 3 was set to Mean Squared Error (MSE). In the regularization term of the evaluation function used in image processing method 3, β = 1 × 10⁻¹⁰ -9 The γ=2 was set. In image processing methods 2 and 3, the input image to the CNN was a 3D random noise image. In image processing methods 2 and 3, the number of iterations was set to 2000, while in image processing method 1, the number of iterations was set to 50.

[0086] Figure 6 shows a cross-sectional image of the brain obtained using image processing method 1. Figure 7 shows a cross-sectional image of the brain obtained using image processing method 2. Figure 8 shows a cross-sectional image of the brain obtained using image processing method 3. The PSNR of the cross-sectional image from image processing method 1 (Figure 6) is 16.50 dB, the PSNR of the cross-sectional image from image processing method 2 (Figure 7) is 19.08 dB, and the PSNR of the cross-sectional image from image processing method 3 (Figure 8) is 19.40 dB. PSNR (Peak Signal to Noise Ratio) is a measure of image quality expressed in decibels (dB), and a higher value indicates better image quality. Compared to image processing methods 1 and 2, image processing method 3 shows a higher PSNR of the cross-sectional image, the implanted tumor is reconstructed with low noise, and the uniformity of the white matter is superior.

[0087] Thus, when training a CNN based on the evaluation results of the error between the computed sinogram and the measured sinogram to create a tomographic image of the subject, it was confirmed that training the CNN using an evaluation function that includes a regularization term representing the evaluation value of the difference between adjacent pixels in the output image from the CNN can suppress image quality degradation due to CNN overfitting and improve noise reduction performance. [Explanation of Symbols]

[0088] 1...Radiation tomography system, 2...Radiation tomography device, 10...Image processing device, 11...Sinogram creation unit, 12...CNN processing unit, 13...Convolution integration unit, 14...Forward projection calculation unit, 15...CNN learning unit.

Claims

1. An image processing apparatus that creates a tomographic image of a subject based on simultaneous counting information collected by a tomography apparatus having multiple detectors arranged around a measurement space in which a subject administered an RI source is placed, A sinogram creation unit that creates a sinogram based on the same-count information collected by the aforementioned tomography device, A CNN processing unit that takes an input image as input to a convolutional neural network and creates an output image using the said convolutional neural network, A forward projection calculation unit that performs forward projection calculations on the output image to create a sinogram, A CNN learning unit uses an evaluation function that includes an error evaluation term representing an evaluation value for the error between the sinogram created by the sinogram creation unit and the sinogram created by the forward projection calculation unit, and a regularization term representing an evaluation value for the difference in pixel values ​​between adjacent pixels in the output image, and trains the convolutional neural network based on the value of this evaluation function. Equipped with, The output image after repeatedly performing the processing of the CNN processing unit, the forward projection calculation unit, and the CNN learning unit is defined as the tomographic image of the subject. Image processing device.

2. The sinogram creation unit creates a sinogram divided into multiple blocks based on the synchronization information collected by the tomography device. The forward projection calculation unit performs a forward projection calculation on the output image to create a sinogram divided into the plurality of blocks, The CNN learning unit trains the convolutional neural network based on the value of the evaluation function for each of the plurality of blocks. The image processing apparatus according to claim 1.

3. The tomographic image, the input image, and the output image are all three-dimensional images. The image processing apparatus according to claim 1.

4. The system further includes a convolution integral unit that performs a convolution integral of the point image distribution function on the output image, The forward projection calculation unit performs forward projection calculation on the output image after processing by the convolution integration unit. The image processing apparatus according to claim 1.

5. The CNN learning unit evaluates the error using the error evaluation term in a region of the sinogram space where simultaneous counting information can be collected by the tomography device. The image processing apparatus according to claim 1.

6. The CNN processing unit inputs an image representing the morphological information of the subject as the input image to the convolutional neural network. The image processing apparatus according to claim 1.

7. The CNN processing unit inputs the MRI image of the subject as the input image to the convolutional neural network. The image processing apparatus according to claim 1.

8. The CNN processing unit inputs the CT image of the subject as the input image to the convolutional neural network. The image processing apparatus according to claim 1.

9. The CNN processing unit inputs the static PET image of the subject as the input image to the convolutional neural network. The image processing apparatus according to claim 1.

10. The CNN processing unit inputs the random noise image as the input image to the convolutional neural network. The image processing apparatus according to claim 1.

11. A tomography system that collects simultaneous counting information and has multiple detectors arranged around a measurement space in which a subject administered with an RI source is placed, The image processing apparatus according to any one of claims 1 to 10, which creates a tomographic image of the subject based on the same-count information collected by the radiographic tomography apparatus, A radiographic tomography system equipped with [specific features / equipment].

12. An image processing method for creating a tomographic image of a subject based on simultaneous counting information collected by a tomography apparatus having multiple detectors arranged around a measurement space in which a subject administered an RI source is placed, A sinogram creation step, which creates a sinogram based on the same-count information collected by the aforementioned tomography device, A CNN processing step involves inputting an input image to a convolutional neural network and creating an output image using the convolutional neural network. A forward projection calculation step to create a sinogram by performing a forward projection calculation on the output image, A CNN learning step in which the convolutional neural network is trained based on the value of an evaluation function, using an evaluation function that includes an error evaluation term representing an evaluation value for the error between the sinogram created in the sinogram creation step and the sinogram created in the forward projection calculation step, and a regularization term representing an evaluation value for the difference in pixel values ​​between adjacent pixels in the output image, Equipped with, The output image after repeatedly performing each of the above CNN processing steps, forward projection calculation steps, and CNN learning steps is defined as the tomographic image of the subject. Image processing methods.

13. In the sinogram creation step, a sinogram divided into multiple blocks is created based on the synchronization information collected by the tomography device. In the forward projection calculation step, the output image is forward-projected to create a sinogram divided into the plurality of blocks. In the CNN learning step, the convolutional neural network is trained based on the value of the evaluation function for each of the plurality of blocks. The image processing method according to claim 12.

14. The tomographic image, the input image, and the output image are all three-dimensional images. The image processing method according to claim 12.

15. The system further includes a convolution integral step in which a convolution integral of the point image distribution function is performed on the output image, In the forward projection calculation step, the output image after processing by the convolution integration step is subjected to forward projection calculation. The image processing method according to claim 12.

16. In the CNN learning step, the error is evaluated using the error evaluation term in the region of the sinogram space where simultaneous counting information can be collected by the tomography device. The image processing method according to claim 12.

17. In the CNN processing step, an image representing the morphological information of the subject is input to the convolutional neural network as the input image. The image processing method according to claim 12.

18. In the CNN processing step, the MRI image of the subject is input to the convolutional neural network as the input image. The image processing method according to claim 12.

19. In the CNN processing step, the CT image of the subject is input to the convolutional neural network as the input image. The image processing method according to claim 12.

20. In the CNN processing step, the static PET image of the subject is input to the convolutional neural network as the input image. The image processing method according to claim 12.

21. In the CNN processing step, a random noise image is input to the convolutional neural network as the input image. The image processing method according to claim 12.

Images