Image processing apparatus and image processing method
By calculating the error evaluation results between the sine graph and the measured sine graph, CNN learns, and using an evaluation function containing error evaluation terms and regularization terms, the image quality degradation caused by CNN over-learning in DIP technology is solved, achieving efficient noise reduction effect.
Patent Information
- Application Number
- CN202380071072.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-10-06
- Filing Date
- 2023-10-02
- Publication Date
- 2025-05-27
AI Technical Summary
Although the noise reduction processing using DIP technology is excellent in noise reduction performance, it has the problem of image quality degradation caused by CNN over-learning.
Based on calculating the error evaluation results between the sine graph and the measured sine graph, CNN is learned, and an evaluation function containing error evaluation terms and regularization terms is used in the CNN learning process to suppress CNN over-learning.
The image quality degradation caused by CNN over-learning is effectively suppressed, and a tomographic image with reduced noise is obtained.
Smart Images
Figure CN120051712A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an apparatus and a method for producing a tomographic image of a subject based on coincidence counting information collected by using a radiation tomography apparatus. Background Art
[0002] As a radiation tomography apparatus capable of acquiring a tomographic image of a subject (living body), a PET (Positron Emission Tomography) apparatus and an SPECT (Single Photon Emission Computed Tomography) apparatus can be cited.
[0003] The PET apparatus includes a detection unit having a large number of small radiation detectors arranged around a measurement space where a subject is placed. The PET apparatus uses the detection unit to detect photon pairs of 511 keV of energy generated by the annihilation of electrons / positrons in a subject administered with a positron-emitting isotope (RI radiation source) by the coincidence counting method, and collects the coincidence counting information thereof.
[0004] Then, based on the large amount of collected coincidence counting information, a tomographic image representing the spatial distribution of the generation frequency of photon pairs in the measurement space (i.e., the spatial distribution of the RI radiation source) can be reconstructed. Such a PET apparatus plays an important role in the field of nuclear medicine and the like, and by using it, for example, research on biological functions and higher functions of the brain can be carried out.
[0005] As a method for reconstructing a tomographic image of a subject based on a large amount of collected coincidence counting information, various methods are known. The image processing method for tomographic image reconstruction described in Non-Patent Document 1 reconstructs a tomographic image by using the Deep Image Prior technique using a convolutional neural network which is a kind of deep neural network. Hereinafter, the convolutional neural network will be referred to as "CNN", and the Deep Image Prior technique will be referred to as "DIP technique".
[0006] The DIP technique utilizes the property of the CNN that meaningful structures in an image are learned faster than random noise (i.e., random noise is difficult to be learned). By using the DIP technique, a tomographic image with reduced noise can be obtained.
[0007] The image processing method described in Non-Patent Document 1 is specifically as follows. A sinogram (hereinafter referred to as "measured sinogram") is created based on a large amount of coincidence counting information collected for the subject. In addition, when an input image (e.g., an MRI image) is input to the CNN, a forward projection calculation (Radon transform) is performed on the image output from the CNN to create a sinogram (hereinafter referred to as "calculated sinogram").
[0008] Then, the error between the calculated sinogram and the measured sinogram is evaluated, and the CNN is made to learn based on the error evaluation result. Using the DIP technique, when repeating the image output from the CNN, the creation of the calculated sinogram using the forward projection calculation, the evaluation of the error, and the learning of the CNN, the calculated sinogram gradually approaches the measured sinogram, and the output image from the CNN gradually approaches the tomographic image of the subject.
[0009] This image processing method includes, on the one hand, the process of forward projection from the CNN output image to the calculated sinogram, and on the other hand, does not include the process of back projection from the measured sinogram to the tomographic image, so that a tomographic image with further reduced noise can be obtained.
[0010] A sinogram is a graph expressed as a histogram representing the frequency (the occurrence frequency of coincidence counting events) of obtaining coincidence counting information in a space (sinogram space) represented by four variables r, θ, z, and δ. The variable r represents the distance from the central axis to the coincidence counting line (the line connecting two detectors that perform coincidence counting of photon pairs). The variable θ represents the azimuth angle of the coincidence counting line. The variable z represents the position in the central axis direction of the midpoint of the coincidence counting line. In addition, the variable δ represents the distance in the central axis direction between the two detectors that perform coincidence counting of photon pairs.
[0011] Prior Art Documents
[0012] Non-Patent Documents
[0013] Non-Patent Document 1: F. Hashimoto, K. Ote and Y. Onishi, "PET Image Reconstruction Incorporating Deep Image Prior and a Forward Projection Model", IEEE Transactions on Radiation and Plasma Medical Sciences, doi:10.1109 / TRPMS.2022.3161569, 2022
[0014] Non-Patent Document 2: J. Nuyts et al., “A concave prior penalizing relative differences for maximum-a-posteriori reconstruction in emission tomography”, IEEE TNS, Vol. 49, Issue 1, pp. 56 - 60, 2002
[0015] Non-Patent Document 3: Hiroyuki Kudo, “Image Reconstruction Methods in Low-Exposure CT - Basics of Statistical Image Reconstruction, Successive Approximation Image Reconstruction, and Compressive Sensing -”, Medical Imaging Technology, Vol. 32, No. 4, pp. 239 - 248, 2014
[0016] Non-Patent Document 4: Antonin Chambolle, “An Algorithm for Total Variation Minimization and Applications”, Journal of Mathematical Imaging and Vision 20, pp. 89 - 97, 2004 Summary of the Invention
[0017] Technical Problem to be Solved by the Invention
[0018] Although the noise reduction processing using the DIP technique has excellent noise reduction performance, there is a problem of image quality degradation caused by overlearning of the CNN. That is, although the DIP technique utilizes the property of the CNN that random noise is not easily learned as described above, as the number of CNN learning increases, the random noise is also restored. Thus, due to the restoration of random noise caused by overlearning of the CNN, the image quality deteriorates.
[0019] An object of the present invention is to provide an image processing apparatus and an image processing method that can suppress image quality degradation caused by overlearning of the CNN in noise reduction processing using the DIP technique and obtain a tomographic image with reduced noise when making a tomographic image of a subject by making the CNN learn based on an evaluation result of the error between a calculated sinogram and a measured sinogram.
[0020] Technical Means for Solving the Problem
[0021] An embodiment of the present invention is an image processing apparatus. The image processing apparatus is an image processing apparatus that creates a tomographic image of a subject based on coincidence count information collected by a radiation tomography apparatus having a plurality of detectors arranged so as to surround a measurement space in which a subject administered with a RI radiation source is placed, and includes: (1) a sinogram creation unit that creates a sinogram based on the coincidence count information collected by the radiation tomography apparatus; (2) a CNN processing unit that inputs an input image to a convolutional neural network and creates an output image by the convolutional neural network; (3) a forward projection calculation unit that performs a forward projection calculation on the output image to create a sinogram; and (4) a CNN learning unit that uses an evaluation function including an error evaluation term representing an evaluation value regarding an error between the sinogram created by the sinogram creation unit and the sinogram created by the forward projection calculation unit and a regularization term representing an evaluation value regarding a difference in pixel values between adjacent pixels in the output image, and causes the convolutional neural network to learn based on the value of the evaluation function, and uses the output image after repeating the processes of the CNN processing unit, the forward projection calculation unit, and the CNN learning unit a plurality of times as the tomographic image of the subject.
[0022] An embodiment of the present invention is a radiation tomography system. The radiation tomography system includes: a radiation tomography apparatus having a plurality of detectors arranged so as to surround a measurement space in which a subject administered with a RI radiation source is placed, and collecting coincidence count information; and an image processing apparatus having the above-described structure that creates a tomographic image of the subject based on the coincidence count information collected by the radiation tomography apparatus.
[0023] An embodiment of the present invention is an image processing method. The image processing method is an image processing method that creates a tomographic image of a subject based on coincidence count information collected by a radiation tomography apparatus having a plurality of detectors arranged so as to surround a measurement space in which a subject administered with a RI radiation source is placed, and includes: (1) a sinogram creation step of creating a sinogram based on the coincidence count information collected by the radiation tomography apparatus; (2) a CNN processing step of inputting an input image to a convolutional neural network and creating an output image by the convolutional neural network; (3) a forward projection calculation step of performing a forward projection calculation on the output image to create a sinogram; and (4) a CNN learning step of using an evaluation function including an error evaluation term representing an evaluation value regarding an error between the sinogram created in the sinogram creation step and the sinogram created in the forward projection calculation step and a regularization term representing an evaluation value regarding a difference in pixel values between adjacent pixels in the output image, and causing the convolutional neural network to learn based on the value of the evaluation function, and using the output image after repeating the processes of the CNN processing step, the forward projection calculation step, and the CNN learning step a plurality of times as the tomographic image of the subject.
[0024] Effects of the Invention
[0025] According to an embodiment of the present invention, when a tomographic image of a subject is created by causing a CNN to learn based on an evaluation result of an error between a calculated sinogram and a measured sinogram, it is possible to suppress deterioration of image quality caused by overlearning of the CNN in noise reduction processing using the DIP technique, and to obtain a tomographic image with reduced noise. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 FIG. is a diagram showing the configuration of the radiation tomographic imaging system 1.
[0027] Figure 2 FIG. is a diagram showing an example of the configuration of the CNN.
[0028] Figure 3 FIG. is a flowchart of the image processing method.
[0029] Figure 4 FIG. is a diagram comparing examples of the calculated sinogram 24 without block division and the calculated sinogram 24 with block division. (a) is a diagram schematically showing the calculated sinogram 24 without block division, and (b) is a diagram schematically showing the calculated sinogram 24 with block division. 1 ~24 16 FIG. is a diagram comparing examples of each of them, (a) is a diagram schematically showing the calculated sinogram 24 without block division, and (b) is a diagram schematically showing the calculated sinogram 24 with block division. 1 ~24 16 FIG.
[0030] Figure 5 FIG. is a diagram for explaining adjacent pixels in the output image.
[0031] Figure 6 FIG. is a diagram showing a tomographic image of the brain obtained by using the image processing method 1.
[0032] Figure 7 FIG. is a diagram showing a tomographic image of the brain obtained by using the image processing method 2.
[0033] Figure 8 FIG. is a diagram showing a tomographic image of the brain using the image processing method 3. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0034] Hereinafter, embodiments of the image processing apparatus and the image processing method will be described in detail with reference to the drawings. In the description of the drawings, the same reference numerals are assigned to the same elements, and redundant description is omitted. The present invention is not limited to these examples, and intends to include all modifications within the meaning and scope represented by the claims and equivalent to the claims.
[0035] Figure 1This is a diagram showing the structure of the tomographic imaging system 1. The tomographic imaging system 1 includes a tomographic imaging device 2 and an image processing device 10. The image processing device 10 includes a sinogram creation unit 11, a CNN processing unit 12, a convolution integration unit 13, a forward projection calculation unit 14, and a CNN learning unit 15.
[0036] In addition, the input image, the output image, and the tomographic image can be either 2D images or 3D images. Hereinafter, these images will be described as 3D images. Furthermore, the measured sinogram and the calculated sinogram can be divided into multiple blocks or not divided into multiple blocks. Hereinafter, the case where these sinograms are divided into multiple blocks will be mainly described.
[0037] The tomographic imaging device 2 is a device that collects coincidence information while reconstructing a tomographic image of a subject. Examples of the tomographic imaging device 2 include a PET device and a SPECT device. Hereinafter, the tomographic imaging device 2 will be described as a PET device.
[0038] The tomographic imaging device 2 includes a detection unit having a plurality of small radiation detectors arranged around a measurement space where the subject is placed. The tomographic imaging device 2 uses the detection unit to detect photon pairs with an energy of 511 keV generated by the annihilation of electrons / positrons in the subject administered with a RI radiation source according to the coincidence counting method, and collects the coincidence information thereof. Then, the tomographic imaging device 2 outputs the collected coincidence information to the image processing device 10.
[0039] The image processing device 10 includes a GPU (Graphics Processing Unit) that performs processing using a convolutional neural network (CNN), an input unit (such as a keyboard and a mouse) that accepts an operator's input, a display unit (such as a liquid crystal display) that displays images, etc., and a storage unit that stores programs and data for executing various processes. As the image processing device 10, for example, a computer having a CPU, a RAM, a ROM, and a hard disk drive, etc. is used.
[0040] The sinogram creation unit 11 creates a measured sinogram 21 based on the coincidence information collected by the tomographic imaging device 2. At this time, the sinogram creation unit 11 creates a measured sinogram 21 divided into multiple (K) blocks. 1 ~21 K 。The measured sinogram 21 k is the measured sinogram of the k-th block among the K blocks. K is an integer of 2 or more, and k is an integer of 1 or more and K or less. The combined sinogram of the divided measured sinograms 21 1 ~21 K is the overall measured sinogram 21.
[0041] The CNN processing unit 12 inputs a three-dimensional input image 20 into the CNN and uses the CNN to produce a three-dimensional output image 22. The three-dimensional input image 20 can be an image representing the morphological information of the subject, an MRI image, a CT image, or a static PET image of the subject, or a random noise image.
[0042] The convolution integration unit 13 performs convolution integration of the point spread function on the three-dimensional output image 22 produced by the CNN processing unit 12 to produce a new three-dimensional output image 23. The point spread function (PSF) is a function representing the response (impulse response) of a tomographic imaging device to a point-line source, and is generally represented by a Gaussian function or an asymmetric Gaussian function with different blurring degrees according to the position within the field of view modeled from the measured data of the point-line source. By providing the convolution integration unit 13, a tomographic image with more excellent image quality can be obtained, and in addition, the stabilization of the learning of the CNN can be achieved.
[0043] The forward projection calculation unit 14 performs forward projection calculation on the three-dimensional output image 23 to produce a calculated sinogram 24. At this time, the forward projection calculation unit 14 produces a calculated sinogram 24 divided into K blocks. 1 ~24 K 。The calculated sinogram 24 k is the calculated sinogram of the k-th block among the K blocks. The combined calculated sinogram 24 1 ~24 K is the overall calculated sinogram 24.
[0044] The block division of the calculated sinogram 24 is performed in the same manner as the block division of the measured sinogram 21. The calculated sinogram 24 of the k-th block k and the measured sinogram 21 of the k-th block k are sinograms of a common area in the entire sinogram space. The method of block division is arbitrary, and it is also possible to perform block division on any one or two or more of the four variables representing the sinogram space. The sizes of the K blocks can be the same or different.
[0045] The CNN learning unit 15 evaluates the error between the measured sinogram 21 k and the calculated sinogram 24 k for each of the K blocks, and makes the CNN learn based on the error evaluation results for each of the K blocks.
[0046] After repeatedly performing the processes of the CNN processing unit 12, the convolution integration unit 13, the forward projection calculation unit 14, and the CNN learning unit 15 respectively, the 3D output image 22 produced by the CNN processing unit 12 is used as the 3D tomographic image of the subject. The 3D output image 23 produced by the convolution integration unit 13 can also be used as the 3D tomographic image of the subject. Since the measured sinogram 21 reflects the response function of the radiation tomography apparatus, it is preferable to use the 3D output image 22 before the convolution integration of the point spread function performed by the convolution integration unit 13 as the 3D tomographic image of the subject.
[0047] In addition, the convolution integration unit 13 can be provided as the final layer of the CNN or separately from the CNN. When the convolution integration unit 13 is provided as the final layer of the CNN, the weight coefficients of the convolution integration unit 13 are maintained constant during the learning of the CNN. In addition, the convolution integration unit 13 may not be provided. When the convolution integration unit 13 is not provided, the forward projection calculation unit 14 performs a forward projection calculation on the 3D output image 22 output from the CNN processing unit 12 to produce a calculated sinogram 24.
[0048] Figure 2 It is a diagram showing an example of the structure of the CNN. The CNN shown in this diagram has a 3D U-net structure including an encoder and a decoder. In this diagram, the number of pixels of the 3D input image 20 input to the CNN is N×N×64, representing the size of each layer of the CNN.
[0049] Figure 3 It is a flowchart of an image processing method. The image processing method includes a sinogram production step S1 performed by the sinogram production unit 11, a CNN processing step S2 performed by the CNN processing unit 12, a convolution integration step S3 performed by the convolution integration unit 13, a forward projection calculation step S4 performed by the forward projection calculation unit 14, and a CNN learning step S5 performed by the CNN learning unit 15.
[0050] In the sinogram production step S1, a measured sinogram 21 divided into K blocks is produced based on the coincidence counting information collected by the radiation tomography apparatus 2. 1 ~21 K In the CNN processing step S2, a 3D input image 20 is input to the CNN, and the CNN produces a 3D output image 22. In the convolution integration step S3, convolution integration of the point spread function is performed on the 3D output image 22 produced in the CNN processing step S2 to produce a new 3D output image 23.
[0051] In the forward projection calculation step S4, a forward projection calculation is performed on the 3D output image 23 to produce a calculated sinogram 24 divided into K blocks. 1 ~24 K. In the CNN learning step S5, the measured sine diagram 21 is evaluated for each of the K blocks k and the calculated sine diagram 24 k The error between them is calculated, and the CNN is made to learn based on the evaluation results of the errors for each of the K blocks.
[0052] After repeatedly performing the processes of the CNN processing step S2, the convolution integration step S3, the forward projection calculation step S4, and the CNN learning step S5 respectively, the three-dimensional output image 22 produced in the CNN processing step S2 is used as the three-dimensional tomographic image of the subject. The three-dimensional output image 23 produced in the convolution integration step S3 can also be used as the three-dimensional tomographic image of the subject. Additionally, the convolution integration step S3 may not be provided.
[0053] Next, the processing contents of each step of the image processing method in the case where the sine diagram is not divided into multiple blocks will be described. In the image processing method where the sine diagram is not divided into blocks, the entire measured sine diagram and the entire calculated sine diagram are processed.
[0054] Hereinafter, let the process performed by the CNN be f, let the three-dimensional input image 20 input to the CNN be z, and let the weight coefficient parameter representing the learning state of the CNN be θ. As the CNN learning progresses, θ changes. Let the three-dimensional output image 22 output from the CNN when the three-dimensional input image z is input to the CNN with the weight coefficient θ be x. The three-dimensional output image x is represented by the following equation (1). In the CNN processing step, the process represented by this equation is performed to produce the three-dimensional output image x.
[0055] [Equation 1]
[0056] x = f(θ|z) (1)
[0057] In the convolution integration step, the three-dimensional output image x produced in the CNN processing step is subjected to convolution integration with the point spread function to produce a new three-dimensional output image x. Additionally, in Figure 1 , the three-dimensional output image x after convolution integration is denoted as PSF(f(θ|z)).
[0058] In the forward projection calculation step, forward projection calculation is performed on the three-dimensional output image x to produce the calculated sine diagram 24. Let the calculated sine diagram 24 be y, and let the projection matrix for performing the forward projection calculation (Radon transform) from the three-dimensional output image x to the calculated sine diagram y be P. The projection matrix is also referred to as the system matrix or the detection probability. The process performed in the forward projection calculation step is represented by the following equation (2).
[0059] [Equation 2]
[0060] y = Px (2)
[0061] In the CNN learning step, let the measured sinogram 21 be y 0 , evaluate the measured sinogram y 0 and the error between the calculated sinogram y (Equation (2) above), and perform CNN learning based on the error evaluation result. The processing performed in the CNN learning step is expressed by the following Equation (3). The constrained optimization problem of this equation becomes a problem of optimizing the CNN parameter θ in such a way that the value of the evaluation function E(y; y 0 ) becomes smaller under the constraint that the three-dimensional output image x produced by the CNN becomes the tomographic image of the subject.
[0062] [Equation 3]
[0063]
[0064] The constrained optimization problem of this Equation (3) can be transformed into an unconstrained optimization problem of the following Equation (4). The evaluation function E can be arbitrary. For example, the L1 norm, the L2 norm, the negative log-likelihood in the Poisson distribution, etc. can be used. When the L2 norm is used as the evaluation function, Equation (4) can be transformed into the following Equation (5).
[0065] [Equation 4]
[0066] min E(Pf(θ|z) - y 0 ) (4)
[0067] [Equation 5]
[0068]
[0069] If the configurations of multiple detectors in the tomographic imaging device are considered, there are cases where there are regions in the sinogram space where simultaneous counting information collection cannot be performed. Therefore, instead of the optimization problem of the above Equation (5), the optimization problem of the following Equation (6) can be set. The m in this Equation (6) is a binary mask function, which is 1 in the regions in the sinogram space where simultaneous counting information collection can be performed, and 0 in the regions where simultaneous counting information collection cannot be performed. Equation (6) is a mathematical formula for selectively evaluating the error in the regions in the sinogram space where simultaneous counting information collection can be performed by taking the Hadamard product of the error (y - y 0 ) and the binary mask function m.
[0070] [Equation 6]
[0071]
[0072] By repeatedly performing the processing of the CNN processing step, the convolution integration step, the forward projection calculation step, and the CNN learning step multiple times, solve this optimization problem for the CNN parameter θ, and calculate the sinogram y to be close to the measured sinogram y0 , the three-dimensional output image x produced by the CNN is close to the tomographic image of the subject.
[0073] Next, the processing contents of each step of the image processing method in the case of dividing the sinogram into blocks will be described in detail. In the case of dividing the sinogram into blocks, in the forward projection calculation step, a forward projection calculation is performed on the three-dimensional output image x to produce a calculated sinogram 24 divided into K blocks 1 ~24 K . Let the calculated sinogram 24 of the k-th block k be y k , and let the projection matrix for performing the forward projection calculation (Radon transform) from the three-dimensional output image x to the calculated sinogram y k be P k . The processing performed in the forward projection calculation step is expressed by the following equation (7).
[0074] [Equation 7]
[0075] y k = P k x (k = 1, 2, 3,..., K) (7)
[0076] In the CNN learning step, let the measured sinogram 21 of the k-th block k be y 0k , and the error between the measured sinogram y 0k and the calculated sinogram y k is evaluated for each of the K blocks, and the CNN is trained based on the error evaluation results for each of the K blocks.
[0077] The processing performed in the CNN learning step is represented by the unconstrained optimization problem of the following equation (8). When the L2 norm is used as the evaluation function, equation (8) can be transformed into the following equation (9). In addition, when selectively evaluating the error in the region in the sinogram space where simultaneous counting information can be collected, it is represented by the unconstrained optimization problem of the following equation (10). m k is the binary mask function in the k-th block.
[0078] [Equation 8]
[0079] min E(P k f(θ|z) - y 0k ) (8)
[0080] [Equation 9]
[0081]
[0082] [Equation 10]
[0083]
[0084] By repeatedly performing the processes of the CNN processing step, the convolution integration step, the forward projection calculation step, and the CNN learning step respectively, the optimization problem is solved with respect to the CNN parameter θ, and thus, for each of the K blocks, the sinogram y is calculated. k Close to the measured sinogram y 0k , the 3D output image x produced by the CNN is close to the tomographic image of the subject.
[0085] Next, regarding the storage capacity required to store data in the RAM of the GPU, a comparison is made between the case where the sinogram is not divided into blocks and the case where it is divided into blocks.
[0086] Generally, a GPU is used in the process using the CNN. The GPU is an arithmetic processing device dedicated to image processing, and has an arithmetic unit and a RAM integrated on one semiconductor chip. It is required that various data used when performing arithmetic processing using the arithmetic unit of the GPU be stored in the RAM of the GPU.
[0087] The data that should be stored in the RAM of the GPU is, for example, the CNN input image, the CNN output image, the weight coefficients indicating the learning state of the CNN, the feature map, the measured sinogram, the calculated sinogram, the parameters required for forward projection calculation, etc., and a large storage capacity is required. However, since the capacity of the RAM of the GPU is limited, in the above-described image processing method, although 2D forward projection calculation can be performed, it is sometimes difficult to perform 3D forward projection calculation.
[0088] Here, let the number of pixels of the 3D output image produced by the CNN be 128×128×64, and let the number of pixels in the sinogram space be 128×128×64×19. In the image processing method in the case of dividing the sinogram into blocks, let K = 16, perform forward projection calculation on the 3D output image, and produce a calculated sinogram 24 that is equally divided into 16 blocks 1 ~24 16 .
[0089] Figure 4 is a diagram showing a comparison of examples of the calculated sinogram 24 in the case of not dividing the sinogram into blocks and the calculated sinogram 24 in the case of dividing the sinogram into blocks 1 ~24 16 respectively. Figure 4 (a) of schematically shows the calculated sinogram 24 in the case of not dividing the sinogram into blocks. Figure 4 (b) of schematically shows the calculated sinogram 24 in the case of dividing the sinogram into blocks 1 ~24 16 .
[0090] The calculated sinogram 24 of each block in the case of performing block division k The number of pixels becomes 128×8×64×19, which is 1 / 16 of the number of pixels of the calculated sinogram 24 in the case of not performing block division. In addition, in the case of performing block division, the projection matrix P k used for performing the forward projection calculation from the 3D output image to the calculated sinogram 24 of the k-th block k has the number of elements that is 1 / 16 of the number of elements of the projection matrix P used for performing the forward projection calculation from the 3D output image to the calculated sinogram 24 in the case of not performing block division.
[0091] In the case of performing block division, the storage capacity required to store the data used in the forward projection calculation can be reduced compared to the case of not performing block division, and these data can be stored in the RAM of the GPU. Therefore, in the case of performing block division, the 3D forward projection calculation from the CNN output image to the calculated sinogram becomes easy, and it is possible to easily make the CNN learn based on the evaluation result of the error between the calculated sinogram and the measured sinogram to create a 3D tomographic image of the subject.
[0092] Next, the evaluation function used by the CNN learning unit 15 in the CNN learning step S5 will be further described. The evaluation functions ((5) formula, (9) formula) described above only include the error evaluation term representing the evaluation value of the error between the measured sinogram y 0 and the calculated sinogram y(=Pf(θ|z)). However, it is preferable to use an evaluation function that includes a regularization term in addition to this error evaluation term. The regularization term is used to suppress overlearning of the CNN and represents the evaluation value of the difference in pixel values between adjacent pixels in the output image.
[0093] That is, instead of the above (5) formula, the evaluation function in the case of not performing block division on the sinogram is the following (11) formula. In addition, instead of the above (9) formula, the evaluation function in the case of performing block division on the sinogram is the following (12) formula.
[0094] [Equation 11]
[0095]
[0096] [Equation 12]
[0097]
[0098] In these mathematical expressions, the first term on the right side is the error evaluation term, and the second term on the right side is the regularization term. This regularization term compensates for the differences in pixel values between adjacent pixels in the output image. β is a hyperparameter that adjusts the degree of the regularization effect. The smaller β is, the smaller the regularization effect. The larger β is, the greater the regularization effect (i.e., the effect of suppressing overfitting in the CNN).
[0099] The regularization term can represent either an evaluation value regarding the differences in pixel values between adjacent pixels in the output image 22 (f(θ|z)) output from the CNN processing unit 12 or an evaluation value regarding the differences in pixel values between adjacent pixels in the output image 23 (PSF(f(θ|z))) output from the convolution integral unit 13.
[0100] In the case of a 2D image, among the pixels adjacent to a certain pixel, there are pixels adjacent in two mutually orthogonal directions, and in addition, there are pixels adjacent in appropriately inclined directions. In the case of a 2D image, except for the pixels located at the edges or corners of the image, the number of pixels adjacent to a certain pixel is 8.
[0101] In the case of a 3D image, among the pixels adjacent to a certain pixel, there are pixels adjacent in three mutually orthogonal directions, and in addition, there are pixels adjacent in appropriately inclined directions. In the case of a 3D image, except for the pixels located at the edges or corners of the image, the number of pixels adjacent to a certain pixel is 26.
[0102] Figure 5 is a diagram illustrating adjacent pixels in the output image. This diagram represents the output image as a 2D image and shows 3×3 pixels therein. Let the pixel value of the pixel at the center in this diagram be λ j , and let the pixel values of the 8 pixels adjacent to this central pixel be λ k (k = 1 to 8). Regarding the differences in pixel values between the pixels adjacent to this central pixel, it is represented by |λ j - λ k |. The regularization term represents the evaluation value regarding the differences in pixel values for all combinations of adjacent pixels in the output image.
[0103] The regularization term only needs to represent the evaluation value regarding the differences in pixel values between adjacent pixels in the output image and can be represented by various mathematical expressions. For example, the regularization term is represented by the following equation (13). In this equation (13), N j represents the set of pixels k adjacent to pixel j. γ represents with respect to the pixel value λ jThe magnitude of the change in the value of the regularization term for the change. Equation (13) has a term in the numerator that is the difference in pixel values of adjacent pixels and a term in the denominator that is the sum of the pixel values of adjacent pixels, representing an evaluation value of the relative difference in pixel values between adjacent pixels in the output image.
[0104] [Equation 13]
[0105]
[0106] In addition, Equation (13) is similar to the mathematical formula described in Non-Patent Document 2. However, in Non-Patent Document 2, the mathematical formula similar to Equation (13) is used in the process of reconstructing the tomographic image of the subject based on the coincidence counting information collected by the PET device, rather than in the process of noise reduction processing of the tomographic image using the DIP technology.
[0107] In addition, as the regularization term, for example, Gibbs prior (Non-Patent Document 3), Total variation (Non-Patent Document 4), etc. can also be used. In addition, these documents also describe the technology for reconstructing the tomographic image of the subject, and do not describe the technology for noise reduction processing of the tomographic image using the DIP technology.
[0108] Next, the results of reconstructing the tomographic image using the digital brain phantom image, producing simulation data using the Monte Carlo simulation of the head PET device, and using this data to reconstruct the tomographic image using Image Processing Methods 1 to 3 respectively are described.
[0109] In Image Processing Method 1, the tomographic image is reconstructed using ML-EM (Maximum Likelihood Expectation Maximization), which is a general image reconstruction method. In Image Processing Method 2, the tomographic image is reconstructed using the evaluation function of Equation (9) in the image processing method described Figures 1 to 4 In Image Processing Method 3, the tomographic image is reconstructed using the evaluation functions of Equation (12) and Equation (13) in the image processing method described Figures 1 to 4 In the image processing method described.
[0110] As the phantom image, a 3D image obtained by embedding a simulated tumor in the white matter part of the brain image obtained from BrainWeb (https: / / brainweb.bic.mni.mcgill.ca / brainweb / ) is used. The number of pixels of the phantom image is 128×128×64. In Image Processing Methods 2 and 3, the number of pixels in the sinogram space is 128×128×64×19, and the sinogram space is equally divided into 2 blocks.
[0111] Let the error evaluation term of the evaluation function used in Image Processing Methods 2 and 3 be the mean squared error (MSE). In the regularization term of the evaluation function used in Image Processing Method 3, β = 1×10 -9 , and γ = 2. Let the input images input to the CNN in Image Processing Methods 2 and 3 be 3D random noise images. Let the number of repetitions in Image Processing Methods 2 and 3 be 2000, and let the number of repetitions in Image Processing Method 1 be 50.
[0112] Figure 6 FIG. is a tomographic image of the brain obtained by using Image Processing Method 1. Figure 7 FIG. is a tomographic image of the brain obtained by using Image Processing Method 2. Figure 8 FIG. is a tomographic image of the brain obtained by using Image Processing Method 3.
[0113] The PSNR of the tomographic image of Image Processing Method 1 ( Figure 6 ) is 16.50 dB, the PSNR of the tomographic image of Image Processing Method 2 ( Figure 7 ) is 19.08 dB, and the PSNR of the tomographic image of Image Processing Method 3 ( Figure 8 ) is 19.40 dB. PSNR (Peak Signal to Noise Ratio) represents the quality of an image in decibels (dB), and the higher the value, the better the image quality. Compared with Image Processing Methods 1 and 2, in Image Processing Method 3, the PSNR of the tomographic image is high, the embedded tumor can be reconstructed with low noise, and the uniformity of the white matter part is excellent.
[0114] In this way, it was confirmed that when making a tomographic image of a subject by enabling the CNN to learn based on the evaluation result of the error between the calculated sinogram and the measured sinogram, using an evaluation function including a regularization term representing an evaluation value regarding the difference in pixel values between adjacent pixels in the output image from the CNN to enable the CNN to learn can suppress the deterioration of image quality caused by overlearning of the CNN, and it was also confirmed that the noise reduction performance can be improved.
[0115] The image processing apparatus and the image processing method are not limited to the above-described embodiments and structural examples, and various modifications can be made.
[0116] The image processing apparatus according to the first aspect of the above-described embodiment is an image processing apparatus that creates a tomographic image of a subject based on coincidence counting information collected by a radiation tomography apparatus including a plurality of detectors arranged so as to surround a measurement space in which a subject administered with a RI radiation source is placed, and includes: (1) a sinogram creation unit that creates a sinogram based on the coincidence counting information collected by the radiation tomography apparatus; (2) a CNN processing unit that inputs an input image to a convolutional neural network and creates an output image by the convolutional neural network; (3) a forward projection calculation unit that performs a forward projection calculation on the output image to create a sinogram; and (4) a CNN learning unit that uses an evaluation function including an error evaluation term representing an evaluation value regarding an error between the sinogram created by the sinogram creation unit and the sinogram created by the forward projection calculation unit and a regularization term representing an evaluation value regarding a difference in pixel values between adjacent pixels in the output image, and causes the convolutional neural network to learn based on the value of the evaluation function, and outputs the output image after repeatedly performing the processes of the CNN processing unit, the forward projection calculation unit, and the CNN learning unit as the tomographic image of the subject.
[0117] In the image processing apparatus according to the second aspect, the following configuration may also be employed: in the configuration according to the first aspect, the sinogram creation unit creates a sinogram divided into a plurality of blocks based on the coincidence counting information collected by the radiation tomography apparatus, the forward projection calculation unit performs a forward projection calculation on the output image to create a sinogram divided into a plurality of blocks, and the CNN learning unit causes the convolutional neural network to learn based on the values of the evaluation functions for the respective plurality of blocks.
[0118] In the image processing apparatus according to the third aspect, the following configuration may also be employed: in the configuration according to the first or second aspect, the tomographic image, the input image, and the output image are each three-dimensional images.
[0119] In the image processing apparatus according to the fourth aspect, the following configuration may also be employed: in the configuration according to any one of the first to third aspects, a convolution integration unit that performs a convolution integration of a point spread function on the output image is further included, and the forward projection calculation unit performs a forward projection calculation on the output image after the process performed by the convolution integration unit.
[0120] In the image processing apparatus according to the fifth aspect, the following configuration may also be employed: in the configuration according to any one of the first to fourth aspects, the CNN learning unit evaluates an error using the error evaluation term in a region in the sinogram space where coincidence counting information collection by the radiation tomography apparatus can be performed.
[0121] In the image processing apparatus according to the sixth aspect, the following configuration may also be employed: in the configuration according to any one of the first to fifth aspects, the CNN processing unit inputs an image representing the morphological information of the subject to the convolutional neural network as the input image.
[0122] In the image processing apparatus according to the seventh mode, the following structure may also be adopted: in the structure of any one of the first to fifth modes, the CNN processing unit inputs the MRI image of the subject as an input image into the convolutional neural network.
[0123] In the image processing apparatus according to the eighth mode, the following structure may also be adopted: in the structure of any one of the first to fifth modes, the CNN processing unit inputs the CT image of the subject as an input image into the convolutional neural network.
[0124] In the image processing apparatus according to the ninth mode, the following structure may also be adopted: in the structure of any one of the first to fifth modes, the CNN processing unit inputs the static PET image of the subject as an input image into the convolutional neural network.
[0125] In the image processing apparatus according to the tenth mode, the following structure may also be adopted: in the structure of any one of the first to fifth modes, the CNN processing unit inputs the random noise image as an input image into the convolutional neural network.
[0126] The tomographic imaging system according to the above-described embodiment includes: a tomographic imaging apparatus having a plurality of detectors arranged to surround a measurement space in which a subject administered with an RI ray source is placed and collecting coincidence counting information; and an image processing apparatus having the above-described structure for creating a tomographic image of the subject based on the coincidence counting information collected by the tomographic imaging apparatus.
[0127] The image processing method according to the first mode of the above-described embodiment is an image processing method for creating a tomographic image of a subject based on coincidence counting information collected by a tomographic imaging apparatus having a plurality of detectors arranged to surround a measurement space in which a subject administered with an RI ray source is placed, and includes: (1) a sinogram creation step of creating a sinogram based on the coincidence counting information collected by the tomographic imaging apparatus; (2) a CNN processing step of inputting an input image into a convolutional neural network and creating an output image by the convolutional neural network; (3) a forward projection calculation step of performing a forward projection calculation on the output image to create a sinogram; and (4) a CNN learning step of using an evaluation function including an error evaluation term representing an evaluation value of an error between the sinogram created in the sinogram creation step and the sinogram created in the forward projection calculation step and a regularization term representing an evaluation value of a difference between pixel values of adjacent pixels in the output image, and causing the convolutional neural network to learn based on the value of the evaluation function, and using the output image after repeatedly performing the processes of the CNN processing step, the forward projection calculation step, and the CNN learning step as the tomographic image of the subject.
[0128] In the image processing method of the second mode, the following structure may also be adopted: In the structure of the first mode, in the sinogram production step, a sinogram divided into a plurality of blocks is produced based on the coincidence counting information collected by the tomographic imaging device, in the forward projection calculation step, forward projection calculation is performed on the output image to produce a sinogram divided into a plurality of blocks, and in the CNN learning step, the convolutional neural network is learned based on the values of the evaluation functions for the respective blocks.
[0129] In the image processing method of the third mode, the following structure may also be adopted: In the structure of the first or second mode, the tomographic image, the input image, and the output image are each three-dimensional images.
[0130] In the image processing method of the fourth mode, the following structure may also be adopted: In the structure of any one of the first to third modes, a convolution integration step of performing convolution integration of the point spread function on the output image is further included, and in the forward projection calculation step, forward projection calculation is performed on the output image after the processing by the convolution integration unit.
[0131] In the image processing method of the fifth mode, the following structure may also be adopted: In the structure of any one of the first to fourth modes, in the CNN learning step, the error is evaluated using an error evaluation term in a region in the sinogram space where coincidence counting information collection by the tomographic imaging device can be performed.
[0132] In the image processing method of the sixth mode, the following structure may also be adopted: In the structure of any one of the first to fifth modes, in the CNN processing step, an image representing the morphological information of the subject is input as the input image to the convolutional neural network.
[0133] In the image processing method of the seventh mode, the following structure may also be adopted: In the structure of any one of the first to fifth modes, in the CNN processing step, the MRI image of the subject is input as the input image to the convolutional neural network.
[0134] In the image processing method of the eighth mode, the following structure may also be adopted: In the structure of any one of the first to fifth modes, in the CNN processing step, the CT image of the subject is input as the input image to the convolutional neural network.
[0135] In the image processing method of the ninth mode, the following structure may also be adopted: In the structure of any one of the first to fifth modes, in the CNN processing step, the static PET image of the subject is input as the input image to the convolutional neural network.
[0136] In the image processing method according to the tenth aspect, the following structure may also be adopted: in the structure of any one of the first to fifth aspects, in the CNN processing step, a random noise image is input as an input image to the convolutional neural network.
[0137] Industrial applicability
[0138] The present invention can be used as an image processing apparatus and an image processing method capable of suppressing image quality deterioration caused by overlearning of a CNN in noise reduction processing using a DIP technique and obtaining a tomographic image with reduced noise when learning the CNN based on an evaluation result of an error between a calculated sinogram and a measured sinogram to produce a tomographic image of a subject.
[0139] Explanation of reference numerals
[0140] 1... Radiation tomography system, 2... Radiation tomography apparatus, 10... Image processing apparatus, 11... Sinogram production unit, 12... CNN processing unit, 13... Convolution integral unit, 14... Forward projection calculation unit, 15... CNN learning unit.
Claims
1. An image processing apparatus, wherein, it is an image processing apparatus that creates a tomographic image of a subject based on coincidence counting information collected by a radiation tomography apparatus having a plurality of detectors, and the plurality of detectors are arranged so as to surround a measurement space in which the subject administered with a RI radiation source is placed, the image processing apparatus includes: a sinogram creation unit that creates a sinogram based on the coincidence counting information collected by the radiation tomography apparatus; a CNN processing unit that inputs an input image into a convolutional neural network and creates an output image by the convolutional neural network; a forward projection calculation unit that performs a forward projection calculation on the output image to create a sinogram; and a CNN learning unit that uses an evaluation function including an error evaluation term representing an evaluation value of an error between the sinogram created by the sinogram creation unit and the sinogram created by the forward projection calculation unit and a regularization term representing an evaluation value of a difference in pixel values between adjacent pixels in the output image, and causes the convolutional neural network to learn based on the value of the evaluation function, and uses the output image after repeatedly performing the processes of the CNN processing unit, the forward projection calculation unit, and the CNN learning unit multiple times as the tomographic image of the subject.
2. The image processing apparatus according to claim 1, wherein, the sinogram creation unit creates a sinogram divided into a plurality of blocks based on the coincidence counting information collected by the radiation tomography apparatus, the forward projection calculation unit performs a forward projection calculation on the output image to create a sinogram divided into the plurality of blocks, and the CNN learning unit causes the convolutional neural network to learn based on the value of the evaluation function for each of the plurality of blocks.
3. The image processing apparatus according to claim 1 or 2, wherein, the tomographic image, the input image, and the output image are each three-dimensional images.
4. The image processing apparatus according to any one of claims 1 to 3, wherein, it further includes a convolution integration unit that performs a convolution integration of a point spread function on the output image, and the forward projection calculation unit performs a forward projection calculation on the output image after the process performed by the convolution integration unit.
5. The image processing apparatus according to any one of claims 1 to 4, wherein, the CNN learning unit evaluates the error using the error evaluation term in a region in the sinogram space where coincidence counting information collection by the radiation tomography apparatus can be performed.
6. The image processing apparatus according to any one of claims 1 to 5, wherein, the CNN processing unit inputs an image representing the morphological information of the subject as the input image into the convolutional neural network.
7. The image processing apparatus according to any one of claims 1 to 5, wherein, the CNN processing unit inputs the MRI image of the subject as the input image into the convolutional neural network.
8. The image processing apparatus according to any one of claims 1 to 5, wherein, the CNN processing unit inputs the CT image of the subject as the input image into the convolutional neural network.
9. The image processing apparatus according to any one of claims 1 to 5, wherein, the CNN processing unit inputs the static PET image of the subject as the input image into the convolutional neural network.
10. The image processing apparatus according to any one of claims 1 to 5, wherein, the CNN processing unit inputs a random noise image as the input image into the convolutional neural network.
11. A tomographic imaging system ， wherein, comprises: a tomographic imaging apparatus having a plurality of detectors arranged to surround a measurement space in which a subject administered with a RI radiation source is placed and collecting coincidence counting information; and the image processing apparatus according to any one of claims 1 to 10 for creating a tomographic image of the subject based on the coincidence counting information collected by the tomographic imaging apparatus.
12. An image processing method, wherein, it is an image processing method for creating a tomographic image of a subject based on coincidence counting information collected by a tomographic imaging apparatus having a plurality of detectors arranged to surround a measurement space in which the subject administered with a RI radiation source is placed, the image processing method includes: a sinogram creation step of creating a sinogram based on the coincidence counting information collected by the tomographic imaging apparatus; a CNN processing step of inputting an input image into a convolutional neural network and creating an output image by the convolutional neural network; a forward projection calculation step of performing a forward projection calculation on the output image to create a sinogram; and a CNN learning step of using an evaluation function including an error evaluation term representing an evaluation value of an error between the sinogram created in the sinogram creation step and the sinogram created in the forward projection calculation step and a regularization term representing an evaluation value of a difference in pixel values between adjacent pixels in the output image, and causing the convolutional neural network to learn based on the value of the evaluation function, The output image after repeatedly performing the processing of the CNN processing step, the forward projection calculation step, and the CNN learning step multiple times is used as the tomographic image of the subject.
13. The image processing method according to claim 12, wherein, in the sinogram creation step, a sinogram divided into a plurality of blocks is created based on the coincidence counting information collected by the tomographic imaging apparatus, in the forward projection calculation step, a forward projection calculation is performed on the output image to create a sinogram divided into the plurality of blocks, in the CNN learning step, the convolutional neural network is caused to learn based on the value of the evaluation function for each of the plurality of blocks.
14. The image processing method according to claim 12 or 13, wherein, the tomographic image, the input image, and the output image are each three-dimensional images.
15. The image processing method according to any one of claims 12 to 14, wherein, it further includes a convolution integration step of performing a convolution integration of a point spread function on the output image, in the forward projection calculation step, a forward projection calculation is performed on the output image after the processing by the convolution integration step.
16. The image processing method according to any one of claims 12 to 15, wherein, in the CNN learning step, the error is evaluated using the error evaluation term in a region in the sinogram space where simultaneous count information collection by the tomographic imaging device can be performed.
17. The image processing method according to any one of claims 12 to 16, wherein, in the CNN processing step, an image representing the morphological information of the subject is input as the input image to the convolutional neural network.
18. The image processing method according to any one of claims 12 to 16, wherein, in the CNN processing step, the MRI image of the subject is input as the input image to the convolutional neural network.
19. The image processing method according to any one of claims 12 to 16, wherein, in the CNN processing step, the CT image of the subject is input as the input image to the convolutional neural network.
20. The image processing method according to any one of claims 12 to 16, wherein, in the CNN processing step, the static PET image of the subject is input as the input image to the convolutional neural network.
21. The image processing method according to any one of claims 12 to 16, wherein, in the CNN processing step, a random noise image is input as the input image to the convolutional neural network.