Image processing apparatus and image processing method

Abstract

A radiation tomography system (1) is provided with a radiation tomography device (2) and an image processing device (10). An image processing device (10) is provided with a sinogram generation unit (11), a CNN processing unit (12), a convolution integration unit (13), an orthographic projection calculation unit (14), and a CNN learning unit (15). An orthographic projection calculation unit (14) performs orthographic projection calculation on a three-dimensional output image (23) and generates calculation sinograms (241-24K) divided into K blocks. A CNN learning unit (15) evaluates the error between the measured sinogram (21k) and the calculated sinogram (24k) for each of the K blocks, and learns the CNN on the basis of the error evaluation results for each of the K blocks. As a result, it is possible to implement an image processing device that can perform three-dimensional orthographic projection calculation from a CNN output image to a computational sinogram, and that can easily generate a three-dimensional tomographic image of a subject by learning the CNN on the basis of an evaluation result of an error between the computational sinogram and a measured sinogram.

Description

Technical Field

[0001] The present disclosure relates to an apparatus and method for generating a three-dimensional tomographic image of a subject based on coincidence counting information collected by a radiation tomography apparatus. Background Art

[0002] Examples of radiation tomography apparatuses that can obtain cross-sectional images of a subject (living body) include PET (Positron Emission Tomography) apparatuses and SPECT (Single Photon Emission Computed Tomography) apparatuses.

[0003] The PET device has a detection unit having a plurality of small radiation detectors arranged around a measurement space where a subject is placed. The PET device detects photon pairs with an energy of 511 keV generated by the annihilation of electrons and positrons in a subject to which a positron emitting isotope (RI ray source) is administered using a coincidence counting method by the detection unit, and collects the coincidence counting information.

[0004] Then, based on the collected multiple simultaneous counting information, a tomographic image showing the spatial distribution of the generation frequency of photon pairs in the measurement space (i.e., the spatial distribution of RI ray sources) can be reconstructed. Such PET devices play an important role in the field of nuclear medicine, etc., and can be used to study, for example, the functions of living organisms or the higher functions of the brain.

[0005] There are various known methods for reconstructing a tomographic image of an object based on a plurality of collected simultaneous counting information. The image processing method for tomographic image reconstruction described in non-patent document 1 reconstructs a tomographic image by using a deep image prior technique of a convolutional neural network, which is a deep neural network. Hereinafter, the convolutional neural network (Convolutional Neural Network) is referred to as "CNN" and the deep image prior technique is referred to as "DIP technique".

[0006] The DIP technique exploits the property of CNN, that is, the meaningful structure in the image is learned faster than random noise (i.e., random noise is difficult to learn). Through the DIP technique, it is possible to obtain a tomographic image with reduced noise.

[0007] The image processing method described in non-patent document 1 is as follows. A sinogram (hereinafter referred to as a "measured sinogram") is generated based on multiple simultaneous counting information collected from the subject. In addition, when an input image (such as an MRI image) is input into a CNN, a forward projection calculation (Radon transform) is performed on the image output from the CNN to generate a sinogram (hereinafter referred to as a "calculated sinogram").

[0008] Then, the error between the calculated sinusoidal graph and the measured sinusoidal graph is evaluated, and the CNN is trained based on the error evaluation result. If the DIP technique is used to repeatedly perform the image output from the CNN, the generation of the calculated sinusoidal graph based on the forward projection calculation, the error evaluation, and the CNN training, the calculated sinusoidal graph gradually approaches the measured sinusoidal graph, and the output image from the CNN gradually approaches the tomographic image of the subject.

[0009] The image processing method includes the processing of forward projection from the CNN output image to the calculated sinusoidal graph, but does not include the processing of back projection from the measured sinusoidal graph to the tomographic image, so it is possible to obtain a tomographic image with further reduced noise.

[0010] The sinogram is a graph represented as a histogram showing the frequency of obtaining simultaneous counting information (the frequency of occurrence of simultaneous counting events) in a space (sinogram space) represented by four variables r, θ, z, and δ. The variable r represents the radial position of the simultaneous counting line (the line connecting the two detectors that simultaneously count photon pairs). The variable θ represents the azimuth angle of the simultaneous counting line. The variable z represents the central axis position of the midpoint of the simultaneous counting line. In addition, the variable δ represents the polar angle of the simultaneous counting line.

[0011] Prior art literature

[0012] Non-patent literature

[0013] Non-patent literature 1: F.Hashimoto, K.Ote and Y.Onishi, "PET Image ReconstructionIncorporating Deep Image Prior and a Forward Projection Model", IEEETransactions on Radiation and Plasma Medical Sciences, doi:10.1109 / TRPMS.2022.3161569, 2022 Summary of the invention

[0014] Problems to be solved by the invention

[0015] Generally, a GPU (Graphics Processing Unit) can be used in processing using CNN. A GPU is a computing device specifically used for image processing, and has a computing unit and RAM (Random Access Memory) integrated on a semiconductor chip. Various data used when computing by the computing unit of the GPU is required to be stored in the RAM of the GPU.

[0016] In the image processing method described in non-patent document 1, data to be stored in the GPU's RAM, such as CNN input images, CNN output images, weight coefficients representing the learning state of CNN, feature maps, measured sinusoidal graphs, calculated sinusoidal graphs, parameters required for orthographic projection calculations, etc., require a huge storage capacity.

[0017] However, since the RAM capacity of a GPU is limited, the image processing method described in Non-Patent Document 1 can perform two-dimensional orthographic projection calculations but cannot perform three-dimensional orthographic projection calculations.

[0018] The object of the present invention is to provide an image processing device and an image processing method that can perform three-dimensional orthographic projection calculation from a CNN output image to a calculated sinusoidal graph, and can enable CNN to learn based on the evaluation result of the error between the calculated sinusoidal graph and the measured sinusoidal graph, thereby easily generating a three-dimensional tomographic image of a subject.

[0019] Means used to solve problems

[0020] An embodiment of the present invention is an image processing device. The image processing device generates a three-dimensional tomographic image of a subject based on coincidence counting information collected by a radiation tomography device, the radiation tomography device having a plurality of detectors arranged to surround a measurement space in which a subject to which an RI radiation source is applied is placed, and includes (1) a sinogram generation unit that generates a sinogram divided into a plurality of blocks based on the coincidence counting information collected by the radiation tomography device; (2) a CNN processing unit that inputs a three-dimensional input image to a convolutional neural network and generates a three-dimensional output image by the convolutional neural network; (3) a forward projection calculation unit that performs forward projection calculation on the three-dimensional output image and generates a sinogram divided into a plurality of blocks; and (4) a CNN learning unit that evaluates the error between the sinogram generated by the sinogram generation unit and the sinogram generated by the forward projection calculation unit for each of the plurality of blocks, performs learning on the convolutional neural network based on the error evaluation results for each of the plurality of blocks, and uses the three-dimensional output image after the processing of the CNN processing unit, the forward projection calculation unit, and the CNN learning unit is repeated multiple times as the three-dimensional tomographic image of the subject.

[0021] An embodiment of the present invention is a radiation tomography system. The radiation tomography system comprises: a radiation tomography apparatus having a plurality of detectors arranged to surround a measurement space in which a subject to be examined is placed and to which an RI ray source is applied, and collecting simultaneous counting information; and an image processing apparatus having the above structure, which generates a three-dimensional tomographic image of the subject based on the simultaneous counting information collected by the radiation tomography apparatus.

[0022] An embodiment of the present invention is an image processing method. The image processing method generates a three-dimensional tomographic image of a subject based on simultaneous counting information collected by a radiation tomography apparatus, the radiation tomography apparatus having a plurality of detectors arranged to surround a measurement space in which a subject to which an RI radiation source is applied is placed, and comprises (1) a sinogram generation step, which generates a sinogram divided into a plurality of blocks based on the simultaneous counting information collected by the radiation tomography apparatus; (2) a CNN processing step, which inputs a three-dimensional input image to a convolutional neural network and generates a three-dimensional output image by the convolutional neural network; (3) a forward projection calculation step, which performs forward projection calculation on the three-dimensional output image and generates a sinogram divided into a plurality of blocks; and (4) a CNN learning step, which evaluates the error between the sinogram generated by the sinogram generation step and the sinogram generated by the forward projection calculation step for each of the plurality of blocks, causes the convolutional neural network to learn based on the error evaluation results for each of the plurality of blocks, and uses the three-dimensional output image after the CNN processing step, the forward projection calculation step, and the CNN learning step are repeatedly performed a plurality of times as the three-dimensional tomographic image of the subject.

[0023] Effects of the Invention

[0024] According to an embodiment of the present invention, as a three-dimensional orthographic projection calculation can be performed from a CNN output image to a calculated sinusoidal graph, the CNN can be learned based on the evaluation result of the error between the calculated sinusoidal graph and the measured sinusoidal graph, thereby easily generating a three-dimensional tomographic image of the subject. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 2 is a diagram showing the configuration of the radiation tomography system 1 .

[0026] Figure 2 This is a diagram showing a structural example of a CNN.

[0027] Figure 3 is a flow chart of the image processing method.

[0028] Figure 4 24 is a graph showing a comparison between the calculated sinusoidal graph 24 in the comparative example and the calculated sinusoidal graph 24 in the present embodiment. 1 ~24 16 Figures of respective examples, Figure 4(a) is a diagram schematically showing a calculated sinusoidal graph 24 in the case of a comparative example, and Figure 4 (b) is a schematic diagram showing the calculated sine diagram 24 in the case of this embodiment. 1 ~24 16 .

[0029] Figure 5 The figures show the cross-sectional images of the brain obtained by the simulations (a) to (c).

[0030] Figure 6 The figures show the cross-sectional images of the brain obtained by the simulations (a) to (c).

[0031] Figure 7 The figures show the cross-sectional images of the brain obtained by the simulations (a) to (c).

[0032] Figure 8 The figures show the cross-sectional images of the brain obtained by the simulations (a) to (c). DETAILED DESCRIPTION

[0033] Hereinafter, embodiments of the image processing device and the image processing method will be described in detail with reference to the accompanying drawings. In addition, in the description of the accompanying drawings, the same elements are marked with the same reference numerals, and repeated descriptions are omitted. The present invention is not limited to these examples, but is indicated by the claims, and is intended to include all changes within the meaning and scope equivalent to the claims.

[0034] Figure 1 1 is a diagram showing a configuration of a radiation tomography system 1. The radiation tomography system 1 includes a radiation tomography apparatus 2 and an image processing apparatus 10. The image processing apparatus 10 includes a sinogram generation unit 11, a CNN processing unit 12, a convolution integration unit 13, a forward projection calculation unit 14, and a CNN learning unit 15.

[0035] The radiation tomography apparatus 2 is an apparatus that collects simultaneous counting information for reconstructing a tomographic image of a subject. Examples of the radiation tomography apparatus 2 include a PET apparatus and a SPECT apparatus. Hereinafter, the radiation tomography apparatus 2 will be described as a PET apparatus.

[0036] The radiation tomography apparatus 2 includes a detection unit having a plurality of small radiation detectors arranged around a measurement space in which a subject is placed. The radiation tomography apparatus 2 detects photon pairs with an energy of 511 keV generated by annihilation of electrons and positrons in a subject to which an RI ray source is applied by using a coincidence counting method in the detection unit, and collects the coincidence counting information. Then, the radiation tomography apparatus 2 outputs the collected coincidence counting information to the image processing apparatus 10.

[0037] The image processing device 10 includes: a GPU that performs processing using CNN; an input unit (such as a keyboard or a mouse) that receives input from an operator; a display unit (such as a liquid crystal display) that displays images, etc.; and a storage unit that stores programs or data for executing various processes. As the image processing device 10, a computer having a CPU, RAM, ROM, a hard disk drive, etc. can be used.

[0038] The sinogram generator 11 generates the measured sinogram 21 based on the coincidence count information collected by the radiation tomography apparatus 2. At this time, the sinogram generator 11 generates the measured sinogram 21 divided into a plurality of (K) blocks. 1 ~21 K . Measured sine diagram 21 k is the measured sinusoidal graph of the kth block among the K blocks. K is an integer greater than 2, and k is an integer greater than 1 and less than K. Combined segmented measured sinusoidal graph 21 1 ~21 K The figure is the overall measured sinusoidal diagram 21.

[0039] The CNN processing unit 12 inputs the three-dimensional input image 20 to the CNN, and generates a three-dimensional output image 22 through the CNN. The three-dimensional input image 20 may be an image representing the morphological information of the subject, an MRI image, a CT image, or a static PET image of the subject, or a random noise image.

[0040] The convolution integration unit 13 performs convolution integration of the point spread function on the three-dimensional output image 22 generated by the CNN processing unit 12, and generates a new three-dimensional output image 23. The point spread function (PSF) is a function that represents the response (impulse response) of the radiation tomography device to a point ray source, and is generally represented by a Gaussian function or an asymmetric Gaussian function whose fuzzy method varies according to the position in the field of view modeled by the measured data of the point ray source. By providing the convolution integration unit 13, a tomographic image with better image quality can be obtained, and the CNN learning can be stabilized.

[0041] The forward projection calculation unit 14 performs forward projection calculation on the three-dimensional output image 23 to generate a calculated sinusoidal map 24. At this time, the forward projection calculation unit 14 generates the calculated sinusoidal map 24 divided into K blocks. 1 ~24 K . Calculate the sine graph 24 k is the calculated sinusoidal graph of the kth block among the K blocks. Combined with the calculated sinusoidal graph of the segmentation 24 1 ~24 K The figure is the overall calculated sinusoidal diagram 24.

[0042] The block division of the calculated sinusoidal graph 24 is performed in the same manner as the block division of the measured sinusoidal graph 21. k And the measured sinusoidal diagram of the kth block 21 k It is a sinogram of a common area in the whole sinogram space. The form of block division is arbitrary, and any one or more than two variables among the four variables representing the sinogram space can be block divided. The sizes of the K blocks can be different or the same.

[0043] CNN learning unit 15 evaluates the measured sinusoidal graph 21 for each of the K blocks k With the calculation of sine graph 24 k The error between them is used to make CNN learn based on the error evaluation results of each of the K blocks.

[0044] After the processing of the CNN processing unit 12, the convolution integration unit 13, the forward projection calculation unit 14 and the CNN learning unit 15 is repeated multiple times, the three-dimensional output image 22 generated by the CNN processing unit 12 is used as the three-dimensional tomographic image of the subject. The three-dimensional output image 23 generated by the convolution integration unit 13 can also be used as the three-dimensional tomographic image of the subject. Since the measured sinogram 21 reflects the response function of the radiation tomography device, it is preferable to use the three-dimensional output image 22 before the convolution integration of the point spread function by the convolution integration unit 13 as the three-dimensional tomographic image of the subject.

[0045] In addition, the convolution integration unit 13 can be set as the final layer of the CNN, or can be set separately from the CNN. When the convolution integration unit 13 is set as the final layer of the CNN, the weight coefficient of the convolution integration unit 13 is maintained constant during the learning of the CNN. In addition, the convolution integration unit 13 may not be set. When the convolution integration unit 13 is not set, the forward projection calculation unit 14 performs forward projection calculation on the three-dimensional output image 22 output from the CNN processing unit 12 to generate a calculated sinogram 24.

[0046] Figure 2 : is a diagram showing an example of the structure of a CNN. The CNN shown in the figure is a three-dimensional U-net structure including an encoder and a decoder. In the figure, the number of pixels of the three-dimensional input image 20 input to the CNN is set to N×N×64, showing the size of each layer of the CNN.

[0047] Figure 3 1 is a flowchart of an image processing method. The image processing method includes a sinogram generation step S1 performed by a sinogram generation unit 11, a CNN processing step S2 performed by a CNN processing unit 12, a convolution integration step S3 performed by a convolution integration unit 13, a forward projection calculation step S4 performed by a forward projection calculation unit 14, and a CNN learning step S5 performed by a CNN learning unit 15.

[0048] In the sinogram generation step S1, based on the coincidence count information collected by the radiation tomography apparatus 2, a measured sinogram 21 divided into K blocks is generated. 1 ~21 K In the CNN processing step S2, a three-dimensional input image 20 is input to the CNN, and the CNN generates a three-dimensional output image 22. In the convolution integration step S3, the three-dimensional output image 22 generated in the CNN processing step S2 is convolved with a point spread function to generate a new three-dimensional output image 23.

[0049] In the forward projection calculation step S4, the three-dimensional output image 23 is subjected to forward projection calculation to generate a calculated sinusoidal image 24 divided into K blocks. 1 ~24 K In the CNN learning step S5, the measured sinusoidal graph 21 is evaluated for each of the K blocks. k With the calculation of sine graph 24 k The error between them is used to make CNN learn based on the error evaluation results of each of the K blocks.

[0050] After repeatedly performing the CNN processing step S2, the convolution integration step S3, the forward projection calculation step S4, and the CNN learning step S5, the three-dimensional output image 22 generated in the CNN processing step S2 is used as the three-dimensional tomographic image of the subject. The three-dimensional output image 23 generated in the convolution integration step S3 can also be used as the three-dimensional tomographic image of the subject. In addition, the convolution integration step S3 may not be provided.

[0051] Next, before describing in detail the processing contents of each step of the image processing method of this embodiment, the processing contents of each step of the image processing method of the comparative example are described. In the image processing method of the comparative example, each measured sinogram and calculated sinogram are not divided into a plurality of blocks.

[0052] In the following, the processing performed by CNN is set to f, the three-dimensional input image 20 input to CNN is set to z, and the weight coefficient parameter representing the learning state of CNN is set to θ. θ changes as the learning of CNN progresses. When the three-dimensional input image z is input to CNN with a weight coefficient of θ, the three-dimensional output image 22 output from CNN is set to x. The three-dimensional output image x is represented by the following formula (1). In the CNN processing step, the processing represented by this formula is performed to generate the three-dimensional output image x.

[0053] [Formula 1]

[0054] x=f(θ|z) (1)

[0055] In the convolution integration step, the convolution integration of the point spread function is performed on the three-dimensional output image x generated in the CNN processing step to generate a new three-dimensional output image x. Figure 1 In the convolution operation, the three-dimensional output image x after convolution is recorded as PSF(f(θ|z)).

[0056] In the forward projection calculation step, the forward projection calculation is performed on the three-dimensional output image x to generate the calculated sinogram 24. The calculated sinogram 24 is set to y, and the projection matrix used for the forward projection calculation (Radon transform) from the three-dimensional output image x to the calculated sinogram y is set to P. The projection matrix is ​​also called a system matrix or a detection probability. The processing performed in the forward projection calculation step is represented by the following formula (2).

[0057] [Formula 2]

[0058] y=Px (2)

[0059] In the CNN learning step, the measured sinusoidal graph 21 is set as y 0 , evaluate the measured sinusoidal diagram y 0 The error between the calculated sinusoidal image y (formula (2) above) and the error evaluation result is used to make the CNN learn. The processing performed in the CNN learning step is represented by the following formula (3). The constrained optimization problem of this formula is to use the error evaluation function E(y; y) under the constraint that the three-dimensional output image x generated by the CNN becomes a tomographic image of the subject. 0 ) is reduced to optimize the CNN parameter θ.

[0060] [Formula 3]

[0061]

[0062] The constrained optimization problem of formula (3) can be transformed into the unconstrained optimization problem of formula (4) below. The error evaluation function E can be arbitrary, for example, the L1 norm, L2 norm, negative log-likelihood in Poisson distribution, etc. can be used. If the L2 norm is used as the error evaluation function, the above formula (4) can be transformed into the following formula (5).

[0063] [Formula 4]

[0064] min E(Pf(θ|z)-y 0 ) (4)

[0065] [Formula 5]

[0066]

[0067] If the configuration of multiple detectors in a radiation tomography device is considered, there are areas in the sinusoidal space where simultaneous counting information cannot be collected. Therefore, the optimization problem of the above formula (5) can be replaced by the optimization problem of the following formula (6). In the formula (6), m is a binary mask function, which takes the value 1 in the area where simultaneous counting information can be collected in the sinusoidal space and takes the value 0 in the area where simultaneous counting information cannot be collected. Formula (6) is obtained by taking the error (yy 0 ) with a binary mask function m to selectively evaluate the error in regions in the sinusoidal pattern space where simultaneous counting information can be collected.

[0068] [Formula 6]

[0069]

[0070] Repeat the CNN processing steps, convolution integration steps, forward projection calculation steps, and CNN learning steps for multiple times. By solving the optimization problem for the CNN parameter θ, the calculated sinusoidal graph y gradually approaches the measured sinusoidal graph y. 0 , and the three-dimensional output image x generated by CNN gradually approaches the tomographic image of the subject.

[0071] Next, the processing contents of each step of the image processing method of this embodiment are described in detail. In this embodiment, in the forward projection calculation step, the three-dimensional output image x is subjected to forward projection calculation to generate a calculated sinusoidal image 24 divided into K blocks. 1 ~24 K . The calculated sinusoidal graph of the kth block 24 k Set to y k , which will be used to calculate the sinusoidal graph y from the three-dimensional output image x k The projection matrix for the forward projection calculation (Radon transform) is set to P k The processing performed in the forward projection calculation step is represented by the following equation (7).

[0072] [Formula 7]

[0073] y k =P k x(k=1,2,3,…,K) (7)

[0074] In the CNN learning step, the measured sinusoidal graph 21k of the kth block is set to y 0k , evaluate the measured sinusoidal graph y for each of the K blocks 0k and calculate the sinusoidal graph y k The error between them is used to make CNN learn based on the error evaluation results of each of the K blocks.

[0075] The processing performed in the CNN learning step is represented by the unconstrained optimization problem of the following formula (8). If the L2 norm is used as the error evaluation function, formula (8) can be transformed into the following formula (9). In addition, when the error is selectively evaluated in the region in the sinusoidal graph space where simultaneous counting information can be collected, it is represented by the unconstrained optimization problem of the following formula (10). k is the binary mask function in the k-th block.

[0076] [Formula 8]

[0077] min E(P k f(θ|z)-y 0k ) (8)

[0078] [Formula 9]

[0079]

[0080] [Formula 10]

[0081]

[0082] By repeatedly performing the CNN processing steps, convolution integration steps, forward projection calculation steps, and CNN learning steps, the optimization problem is solved for the CNN parameters θ, thereby calculating the sinusoidal graph y for each of the K blocks. k Gradually approaching the measured sinusoidal graph y 0k , the three-dimensional output image x generated by CNN gradually approaches the tomographic image of the subject.

[0083] Next, a comparison between the comparative example and the present embodiment regarding the storage capacity required for storing data in the RAM of the GPU is described. Here, the number of pixels of the three-dimensional output image generated by the CNN is set to 128×128×64, and the number of pixels of the sinogram space is set to 128×128×64×19. In the image processing method of the present embodiment, with K=16, the three-dimensional output image is subjected to forward projection calculation to generate a calculated sinogram 24 divided equally into 16 blocks. 1 ~24 16 .

[0084] Figure 4 24 is a graph showing a comparison between the calculated sine graph 24 in the comparative example and the calculated sine graph 24 in the present embodiment. 1 ~24 16 Figures of the respective examples. Figure 4 (a) schematically shows a calculated sinusoidal graph 24 in the case of the comparative example. Figure 4 (b) schematically shows the calculated sinusoidal diagram 24 in the case of this embodiment. 1 ~2416 .

[0085] In the case of this embodiment, the calculated sinusoidal diagram 24 of each block k The number of pixels of is 128×8×64×19, which is 1 / 16 of the number of pixels of the calculated sinogram 24 in the case of the comparative example. In addition, in the case of this embodiment, the calculated sinogram 24 for performing the calculation from the three-dimensional output image to the k-th block is k The projection matrix P calculated by the orthographic projection k The number of elements of is 1 / 16 of the number of elements of the projection matrix P used for performing the forward projection calculation from the three-dimensional output image to the calculated sinusoidal image 24 in the case of the comparative example.

[0086] In this embodiment, compared with the comparative example, the storage capacity required for storing the data used in the orthographic projection calculation can be reduced, and the data can be stored in the RAM of the GPU. Therefore, in this embodiment, the three-dimensional orthographic projection calculation from the CNN output image to the calculated sinogram can be performed, and the CNN can be learned based on the evaluation result of the error between the calculated sinogram and the measured sinogram, thereby easily generating a three-dimensional tomographic image of the subject.

[0087] Next, the simulation data generated by Monte Carlo simulation of a head PET device using a digital brain phantom image is described, and the tomographic image is obtained by the image processing method of this embodiment and the ML-EM method using the simulation data. The phantom image is obtained from BrainWeb (https: / / brainweb.bic.mni.mcgill.ca / brainweb / ). The ML-EM (Maximum Likelihood Expectation Maximization) method is a common image reconstruction method.

[0088] In the image processing method of this embodiment, the three-dimensional input image input to CNN is set to a random noise image, the number of pixels of the input image is set to 128×128×64, the number of pixels of the three-dimensional output image generated by CNN is set to 128×128×64, the number of pixels of the sinogram space is set to 128×128×64×19, the sinogram space is equally divided into 16 blocks, and the error evaluation function is set to the negative log-likelihood in the Poisson distribution. In the image processing method of this embodiment, the number of repetitions is set to 2000, and in the ML-EM method, the number of repetitions is set to 50.

[0089] Figures 5 to 8 The figures show the tomographic images of the brain obtained by simulation. These figures show the tomographic images of the cross section at four different positions in the body axis direction in the three-dimensional tomographic image. Figures 5 to 8 In the figure, (a) represents a phantom image (correct image), (b) represents a tomographic image obtained by the ML-EM method, and (c) represents a tomographic image obtained by the image processing method of this embodiment.

[0090] Compared with the tomographic image obtained by the ML-EM method, the image processing method of this embodiment can obtain a tomographic image with significantly better image quality. In addition, through this simulation, it can be confirmed that in this embodiment, the three-dimensional positive projection calculation from the CNN output image to the calculated sinusoidal graph can be performed by the GPU, and the CNN can be learned based on the evaluation result of the error between the calculated sinusoidal graph and the measured sinusoidal graph, thereby generating a three-dimensional tomographic image of the subject.

[0091] The image processing device and the image processing method are not limited to the above-described embodiments and configuration examples, and various modifications are possible.

[0092] According to the first mode of the above-mentioned embodiment, an image processing device generates a three-dimensional tomographic image of a subject based on coincidence counting information collected by a radiation tomography device, the radiation tomography device having a plurality of detectors arranged to surround a measurement space in which a subject to which an RI ray source is applied is placed, and comprises (1) a sinogram generation unit that generates a sinogram divided into a plurality of blocks based on the coincidence counting information collected by the radiation tomography device; (2) a CNN processing unit that inputs a three-dimensional input image to a convolutional neural network and generates a three-dimensional output image by the convolutional neural network; (3) a forward projection calculation unit that performs forward projection calculation on the three-dimensional output image and generates a sinogram divided into a plurality of blocks; and (4) a CNN learning unit that evaluates the error between the sinogram generated by the sinogram generation unit and the sinogram generated by the forward projection calculation unit for each of the plurality of blocks, performs learning on the convolutional neural network based on the error evaluation results for each of the plurality of blocks, and uses the three-dimensional output image after the processing of the CNN processing unit, the forward projection calculation unit, and the CNN learning unit is repeated multiple times as the three-dimensional tomographic image of the subject.

[0093] In the image processing device of the second mode, it can also be constructed that, in the structure of the first mode, there is also a convolution integration unit, which performs convolution integration of the point spread function on the three-dimensional output image, and the forward projection calculation unit performs forward projection calculation on the three-dimensional output image processed by the convolution integration unit.

[0094] In the image processing device of the third aspect, in the configuration of the first or second aspect, the CNN learning unit may evaluate the error in a region in the sinogram space where the coincidence counting information can be collected by the radiation tomography apparatus.

[0095] In the image processing device of the fourth aspect, in any one of the structures of the first to third aspects, the CNN processing unit may input an image representing morphological information of the subject as a three-dimensional input image into the convolutional neural network.

[0096] In the image processing device of the fifth aspect, in any one of the configurations of the first to third aspects, the CNN processing unit may input the MRI image of the subject as a three-dimensional input image into the convolutional neural network.

[0097] In the image processing device of the sixth aspect, in any one of the structures of the first to third aspects, the CNN processing unit may input the CT image of the subject as a three-dimensional input image into the convolutional neural network.

[0098] In the image processing device of the seventh aspect, in any one of the configurations of the first to third aspects, the CNN processing unit may input a static PET image of the subject as a three-dimensional input image into the convolutional neural network.

[0099] In the image processing device of the eighth aspect, it can also be configured that, in any one of the structures of the first to third aspects, the CNN processing unit inputs the random noise image as the three-dimensional input image to the convolutional neural network.

[0100] According to the above-mentioned embodiment, the radiation tomography system comprises: a radiation tomography apparatus having a plurality of detectors arranged to surround a measurement space in which a subject to be subjected to an RI ray source is placed, and collecting simultaneous counting information; and an image processing apparatus of the above-mentioned structure, which generates a three-dimensional tomographic image of the subject based on the simultaneous counting information collected by the radiation tomography apparatus.

[0101] According to the first mode of the above-mentioned embodiment, an image processing method generates a three-dimensional tomographic image of a subject based on coincidence counting information collected by a radiation tomography apparatus, the radiation tomography apparatus having a plurality of detectors arranged to surround a measurement space in which a subject to which an RI ray source is applied is placed, and comprises (1) a sinogram generation step of generating a sinogram divided into a plurality of blocks based on the coincidence counting information collected by the radiation tomography apparatus; (2) a CNN processing step of inputting a three-dimensional input image to a convolutional neural network and generating a three-dimensional output image by the convolutional neural network; (3) a forward projection calculation step of performing forward projection calculation on the three-dimensional output image and generating a sinogram divided into a plurality of blocks; and (4) a CNN learning step of evaluating the error between the sinogram generated by the sinogram generation step and the sinogram generated by the forward projection calculation step for each of the plurality of blocks, performing learning on the convolutional neural network based on the error evaluation results for each of the plurality of blocks, and using the three-dimensional output image after the CNN processing step, the forward projection calculation step, and the CNN learning step are each repeatedly performed a plurality of times as the three-dimensional tomographic image of the subject.

[0102] In the image processing method of the second mode, it can also be constructed that, in the structure of the first mode, there is also a convolution integration step, which performs a convolution integration of the point spread function on the three-dimensional output image, and in the forward projection calculation step, the forward projection calculation is performed on the three-dimensional output image processed by the convolution integration step.

[0103] In the image processing method of the third aspect, in the configuration of the first or second aspect, in the CNN learning step, the error is evaluated in a region in the sinogram space where coincidence count information can be collected by the radiation tomography apparatus.

[0104] In the image processing method of the fourth aspect, in any one of the structures of the first to third aspects, in the CNN processing step, an image representing morphological information of the subject is input as a three-dimensional input image to the convolutional neural network.

[0105] In the image processing method of the fifth aspect, in any one of the structures of the first to third aspects, in the CNN processing step, the MRI image of the subject is input as a three-dimensional input image to the convolutional neural network.

[0106] In the image processing method of the sixth aspect, in any one of the structures of the first to third aspects, in the CNN processing step, the CT image of the subject is input as a three-dimensional input image to the convolutional neural network.

[0107] In the image processing method of the seventh aspect, in any one of the structures of the first to third aspects, in the CNN processing step, a static PET image of the subject is input to the convolutional neural network as a three-dimensional input image.

[0108] In the image processing method of the eighth aspect, it can also be configured that in any one of the structures of the first to third aspects, in the CNN processing step, the random noise image is input into the convolutional neural network as a three-dimensional input image.

[0109] Industrial Applicability

[0110] The present invention can be used as an image processing device and an image processing method that can perform three-dimensional orthographic projection calculation from a CNN output image to a calculated sinusoidal graph, and can enable CNN to learn based on the evaluation result of the error between the calculated sinusoidal graph and the measured sinusoidal graph, thereby easily generating a three-dimensional tomographic image of the subject.

[0111] Description of Reference Numerals

[0112] 1: Radiation tomography system, 2: Radiation tomography apparatus, 10: Image processing apparatus, 11: Sine graph generation unit, 12: CNN processing unit, 13: Convolution integration unit, 14: Forward projection calculation unit, 15: CNN learning unit.

Claims

1. An image processing device, wherein: generating a three-dimensional tomographic image of the subject based on the coincidence counting information collected by a radiation tomography apparatus having a plurality of detectors arranged to surround a measurement space in which the subject to which an RI ray source is applied is placed, have: a sinogram generating unit for generating a sinogram divided into a plurality of blocks based on the coincidence counting information collected by the radiation tomography apparatus; A CNN processing unit, which inputs a three-dimensional input image to a convolutional neural network and generates a three-dimensional output image through the convolutional neural network; an orthographic projection calculation unit, which performs orthographic projection calculation on the three-dimensional output image and generates a sinogram divided into the plurality of blocks; as well as a CNN learning unit that evaluates, for each of the plurality of blocks, an error between a sinogram generated by the sinogram generating unit and a sinogram generated by the forward projection calculating unit, and causes the convolutional neural network to learn based on the error evaluation results for each of the plurality of blocks; The three-dimensional output image after the processing by the CNN processing unit, the forward projection calculation unit, and the CNN learning unit is repeatedly performed a plurality of times is used as a three-dimensional tomographic image of the subject.

2. The image processing device according to claim 1, wherein: The method further comprises a convolution integration unit for performing convolution integration of a point spread function on the three-dimensional output image. The forward projection calculation unit performs forward projection calculation on the three-dimensional output image processed by the convolution integration unit.

3. The image processing device according to claim 1 or 2, wherein: The CNN learning section evaluates the error in a region in a sinogram space in which coincidence count information can be collected by the radiation tomography apparatus.

4. The image processing device according to any one of claims 1 to 3, wherein: The CNN processing unit inputs an image representing morphological information of the subject as the three-dimensional input image to the convolutional neural network.

5. The image processing device according to any one of claims 1 to 3, wherein: The CNN processing unit inputs the MRI image of the subject as the three-dimensional input image into the convolutional neural network.

6. The image processing device according to any one of claims 1 to 3, wherein: The CNN processing unit inputs the CT image of the subject as the three-dimensional input image into the convolutional neural network.

7. The image processing device according to any one of claims 1 to 3, wherein: The CNN processing unit inputs the static PET image of the subject as the three-dimensional input image to the convolutional neural network.

8. The image processing device according to any one of claims 1 to 3, wherein: The CNN processing unit inputs a random noise image as the three-dimensional input image to the convolutional neural network.

9. A radiation tomography system, wherein: have: A radiation tomography apparatus having a plurality of detectors arranged to surround a measurement space in which a subject to which an RI ray source is applied is placed, and collecting coincidence counting information; and An image processing apparatus according to any one of claims 1 to 8, which generates a three-dimensional tomographic image of the subject based on the coincidence counting information collected by the radiation tomography apparatus.

10. An image processing method, wherein: generating a three-dimensional tomographic image of the subject based on the coincidence counting information collected by a radiation tomography apparatus having a plurality of detectors arranged to surround a measurement space in which the subject to which an RI ray source is applied is placed, have: a sinogram generating step of generating a sinogram divided into a plurality of blocks based on the coincidence counting information collected by the radiation tomography apparatus; A CNN processing step, which inputs a three-dimensional input image to a convolutional neural network and generates a three-dimensional output image through the convolutional neural network; a forward projection calculation step, which performs a forward projection calculation on the three-dimensional output image and generates a sinusoidal graph divided into the plurality of blocks; as well as a CNN learning step of evaluating, for each of the plurality of blocks, an error between the sinusoidal graph generated in the sinusoidal graph generating step and the sinusoidal graph generated in the forward projection calculating step, and causing the convolutional neural network to learn based on the error evaluation results for each of the plurality of blocks, The three-dimensional output image after the CNN processing step, the forward projection calculation step, and the CNN learning step are repeatedly performed a plurality of times is used as a three-dimensional tomographic image of the subject.

11. The image processing method according to claim 10, wherein: There is also a convolution step, which convolves the three-dimensional output image with a point spread function, In the forward projection calculation step, a forward projection calculation is performed on the three-dimensional output image processed by the convolution integration step.

12. The image processing method according to claim 10 or 11, wherein: In the CNN learning step, the error is evaluated in a region in a sinogram space in which coincidence count information can be collected by the radiation tomography apparatus.

13. The image processing method according to any one of claims 10 to 12, wherein: In the CNN processing step, an image representing morphological information of the subject is input into the convolutional neural network as the three-dimensional input image.

14. The image processing method according to any one of claims 10 to 12, wherein: In the CNN processing step, the MRI image of the subject is input into the convolutional neural network as the three-dimensional input image.

15. The image processing method according to any one of claims 10 to 12, wherein: In the CNN processing step, the CT image of the subject is input into the convolutional neural network as the three-dimensional input image.

16. The image processing method according to any one of claims 10 to 12, wherein: In the CNN processing step, the static PET image of the subject is input into the convolutional neural network as the three-dimensional input image.

17. The image processing method according to any one of claims 10 to 12, wherein: In the CNN processing step, a random noise image is input into the convolutional neural network as the three-dimensional input image.

Images