Image processing device and image processing method

DE112023000780T5Pending Publication Date: 2025-03-27HAMAMATSU PHOTONICS KK
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
DE112023000780
Authority / Receiving Office
DE · DE
Patent Type
Applications
Current Assignee / Owner
Priority Date
2022-02-02
Filing Date
2023-01-30
Publication Date
2025-03-27

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

An image processing method generates a tomographic image of an object from a given initial state by repeatedly executing a reconstruction step S1, a CNN processing step S2, and an update step S3 a multitude of times. In the reconstruction step S1, a first image is generated by processing using an iterative list-mode reconstruction method. In the CNN processing step S2, input information is fed into a CNN, and a second image is generated by the CNN using a DIP technique, thus training the CNN. In the update step S3, a third image is updated based on the first and second images.In this way, a device and a method are realized that are capable of generating a tomographic image in which noise is reduced based on list data collected by a radiation tomography device.
Need to check novelty before this filing date? Find Prior Art

Description

Technical area

[0001] The present disclosure relates to an apparatus and method for generating a tomographic image based on list data collected by a radiation tomography apparatus. State of the art

[0002] Radiation tomography devices that can take a tomographic image of an object (living body) include a positron emission tomography (PET) device and a single-photon emission computed tomography (SPECT) device.

[0003] The PET device comprises a detection unit with a large number of small radiation detectors arranged around a measurement space containing the object. The PET device detects a photon pair with an energy of 511 keV generated by electron-positron annihilation in the object into which a positron-emitting isotope (RI source) is introduced by a coincidence method using the detection unit, and collects coincidence information therefrom. A tomographic image representing a spatial distribution of the generation frequency of the photon pairs in the measurement space (i.e., a spatial distribution of the RI sources) can then be reconstructed based on the numerous collected coincidence information, as described above.

[0004] In this case, a dynamic PET image including the tomographic images of a plurality of frames can be obtained by dividing list data in which the coincidence information collected by the PET device is arranged in time series into the plurality of frames in a collection order, and performing image reconstruction processing from the list data using a data group contained in each frame. The PET device plays an important role in nuclear medicine and the like and can be used to study, for example, a biological function or a higher-order brain function.

[0005] The tomographic image reconstructed as described above contains a lot of noise, and therefore, noise removal processing using an image filter is necessary. Examples of image filters used for noise removal include a Gaussian filter and a guided filter. The Gaussian filter is commonly used. The guided filter, on the other hand, was only developed in recent years and differs from the Gaussian filter in its ability to preserve a boundary of the shading in the image.

[0006] Furthermore, a technique for removing noise in tomographic images through a deep image preparation technique using a convolutional neural network, which is a type of deep neural network, has been proposed (Non-Patent Document 1). Hereinafter, the deep neural network is referred to as "DNN," the convolutional neural network is referred to as "CNN," and the deep image preparation technique is referred to as "DIP technique."

[0007] In the DIP technique, the noise in the object image can be reduced by exploiting the property of the CNN that a meaningful structure in the object image is learned faster than random noise (i.e., the random noise is learned with less probability).

[0008] In each of the denoising techniques described above, the tomographic image is created using a histogram-mode reconstruction method using the list data, and then the tomographic image is processed to reduce noise. Furthermore, in some cases, the denoising technique can be incorporated into the histogram-mode reconstruction method as a regularization.

[0009] In the histogram-mode reconstruction method, a histogram representing the number of coincidence events detected by each detector pair is created from the list data, and the tomographic image is reconstructed based on the above histogram. The histogram format uses, for example, a four-dimensional array (three-dimensional sinogram) consisting of radius × body axis × azimuth × inclination angle.

[0010] Furthermore, with the development of radiation tomography technology in recent years, when new information such as detection time difference (TOF) information of a pair of radiation detectors, photon interaction depth (DOI) in the detector, and the like are added to the list data, the array format of the histogram becomes high-dimensional, such as five dimensions or six dimensions. As a result, in the histogram-mode reconstruction method in which the tomographic image is reconstructed from the list data by the histogram, the load on an operating unit including a CPU and the like for performing a series of the above-mentioned processes becomes excessively high.

[0011] An iterative list-mode reconstruction method has been proposed as a technique capable of solving the problem of the histogram-mode reconstruction method described above (Non-Patent Document 2). In the iterative list-mode reconstruction method, the tomographic image is reconstructed directly from the list data (without traversing the histogram) through repeated iterative approximation. Citation listNon-patent literature Non-Patent Document 1: Kuang Gong et al. “PET Image Reconstruction Using Deep Image Prior,” IEEE Transactions on Medical Imaging, December 2018 Non-Patent Document 2: AJ Reader et al., “One-Pass List-Mode EM Algorithm for High-Resolution 3-D PET Image Reconstruction Into Large Arrays,” IEEE Transactions on Nuclear Science, Vol.49, No.3, pp.693-699, 2002 Summary of the inventionTechnical problem

[0012] Many noise removal techniques have been researched and developed for the histogram-mode reconstruction method. However, the iterative list-mode reconstruction method has the problem that it is difficult to directly process the list data using the CNN. Therefore, a technique for the iterative list-mode reconstruction method that can effectively remove noise by incorporating the CNN is not known.

[0013] An object of the present invention is to provide an image processing apparatus and an image processing method capable of generating a tomographic image in which noise is reduced based on list data collected by a radiation tomography apparatus. Solution to the problem

[0014] A first aspect of the present invention is an image processing device. The image processing device is an image processing device for generating a tomographic image based on list data collected by a radiation tomography device, and comprises (1) a reconstruction unit for regenerating a first image by repeatedly processing an image obtained by updating the first image based on the list data using an iterative list-mode reconstruction method near a difference between a second image and a third image; (2) a CNN processing unit for inputting input information to a convolutional neural network, generating the second image by the convolutional neural network, and training the convolutional neural network.to bring the generated second image close to a sum of the first image and the third image, and (3) an updating unit for updating the third image based on the first image and the second image, wherein, starting from an initial state of a training state of the convolutional neural network, the first image, the second image, and the third image, the generation of the first image by the reconstruction unit, the generation of the second image and the training of the convolutional neural network by the CNN processing unit, and the updating of the third image by the updating unit are repeatedly performed, and the first image or the second image obtained by the repeated processing is set as the tomographic image.

[0015] A second aspect of the present invention is an image processing device. The image processing device is an image processing device for generating a tomographic image based on list data collected by a radiation tomography device, and includes (1) a reconstruction unit for generating a first image by updating a third image based on the list data using an iterative list-mode reconstruction method; (2) a CNN processing unit for inputting input information to a convolutional neural network, generating a second image through the convolutional neural network, and training the convolutional neural network to bring the generated second image close to the third image;and (3) an updating unit for updating the third image based on the first image and the second image, wherein, starting from an initial state of a training state of the convolutional neural network and the third image, the generation of the first image by the reconstruction unit, the generation of the second image and the training of the convolutional neural network by the CNN processing unit, and the updating of the third image by the updating unit are repeatedly performed, and any one of the first image, the second image, and the third image obtained by repeated processing is set as the tomographic image.;

[0016] One embodiment of the present invention is a radiation tomography system. The radiation tomography system includes a radiation tomography device for collecting list data for reconstructing a tomographic image of an object; and the image processing device of the above configuration for generating the tomographic image based on the list data collected by the radiation tomography device.

[0017] A first aspect of the present invention is an image processing method. The image processing method is an image processing method for generating a tomographic image based on list data collected by a radiation tomography apparatus, and comprises (1) a reconstruction step of regenerating a first image by repeatedly performing processing on an image obtained by updating the first image based on the list data using an iterative list-mode reconstruction method near a difference between a second image and a third image;(2) a CNN processing step of inputting input information to a convolutional neural network, generating the second image by the convolutional neural network, and training the convolutional neural network to make the generated second image close to a sum of the first image and the third image;and (3) an updating step of updating the third image based on the first image and the second image, wherein, starting from an initial state of each training state of the convolutional neural network, the first image, the second image, and the third image, the generation of the first image in the reconstruction step, the generation of the second image and the training of the convolutional neural network in the CNN processing step, and the updating of the third image in the updating step are repeatedly performed, and the first image or the second image obtained by the repeated processing is set as the tomographic image.

[0018] A second aspect of the present invention is an image processing method. The image processing method is an image processing method for generating a tomographic image based on list data collected by a radiation tomography apparatus, and includes (1) a reconstruction step of generating a first image by updating a third image based on the list data using an iterative list-mode reconstruction method; (2) a CNN processing step of inputting input information to a convolutional neural network, generating a second image by the convolutional neural network, and training the convolutional neural network to make the generated second image close to the third image;and (3) an updating step of updating the third image based on the first image and the second image, and starting from an initial state of each training state of the convolutional neural network and the third image, the generation of the first image in the reconstruction step, the generation of the second image and the training of the convolutional neural network in the CNN processing step, and the updating of the third image in the updating step are repeatedly performed, and any one of the first image, the second image, and the third image obtained by repeated processing is set as the tomographic image.; Advantageous effects of the invention

[0019] According to aspects of the present invention, a tomographic image in which noise is reduced can be created based on list data collected by a radiation tomography apparatus. Brief description of the drawings [ Fig. 1] Fig. 1 is a diagram showing the configuration of a radiation tomography system 1. [ Fig. 2] Fig. 2 is a flowchart illustrating an image processing method. [ Fig. 3] Fig. 3 is a diagram showing a flow of the image processing method according to a first aspect. [ Fig. 4] Fig. Figure 4 contains diagrams each showing a phantom image (correct image) and shows (a) an image of a transverse section and (b) an image of the transverse section. [ Fig. 5] Fig. Figure 5 contains diagrams each showing the phantom image (correct image) and shows (a) an image of a coronal section and (b) an image of a sagittal section. [ Fig. 6] Fig. 6 includes diagrams each showing a tomographic image obtained by the image processing method according to a first comparative example, and shows (a) a transverse section image and (b) a transverse section image. [ Fig. 7] Fig. 7 includes diagrams each showing the tomographic image obtained by the image processing method according to the first comparative example, and shows (a) a coronal section image and (b) a sagittal section image. [ Fig. 8] Fig. 8 includes diagrams each showing the tomographic image obtained by the image processing method according to a second comparative example, and shows (a) an image of the transverse section and (b) an image of the transverse section. [ Fig. 9] Fig. 9 includes diagrams each showing the tomographic image obtained by the image processing method according to the second comparative example, and shows (a) a coronal section image and (b) a sagittal section image. [ Fig. 10] Fig. 10 includes diagrams each showing the tomographic image obtained by the image processing method according to an example, and shows (a) an image of the transverse section and (b) an image of the transverse section. [ Fig. 11] Fig. 11 includes diagrams each showing the tomographic image obtained by the image processing method according to the example, and shows (a) a coronal section image and (b) a sagittal section image. [ Fig. 12] Fig. 12 is a diagram showing the PSNR of the tomographic image obtained by each of the first and second comparative examples and the example. [ Fig. 13] Fig. 13 is a diagram showing a CRC of the tomographic image obtained by each of the first and second comparative examples and the example. [ Fig. 14] Fig. 14 is a diagram showing a flow of the image processing method according to a second aspect. Description of the embodiments

[0020] Embodiments of an image processing apparatus and an image processing method will be described in detail below with reference to the accompanying drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant descriptions are omitted. The present invention is not limited to these examples.

[0021] Fig. Figure 1 is a diagram showing the configuration of a radiation tomography system 1. The radiation tomography system 1 includes a radiation tomography device 2 and an image processing device 10. The image processing device 10 includes a reconstruction unit 11, a CNN processing unit 12, an update unit 13, and a storage unit 14.

[0022] A computer including a CPU, a RAM, a ROM, a hard disk drive, and the like is used as the image processing device 10. Furthermore, the image processing device 10 includes an input unit (e.g., a keyboard or a mouse) for receiving an operator's input, and a display unit (e.g., a liquid crystal display) for displaying an image and the like.

[0023] The radiation tomography device 2 is a device for acquiring list data for reconstructing a tomographic image of an object. Examples of the radiation tomography device 2 include a PET device and a SPECT device. In the following description, it is assumed that the radiation tomography device 2 is a PET device.

[0024] The radiation tomography device 2 includes a detection unit with a large number of small radiation detectors arranged around a measurement space in which the object is located. The radiation tomography device 2 detects a photon pair with an energy of 511 keV generated by electron-positron annihilation in the object into which a positron-emitting isotope (RI source) is injected by a coincidence method using the detection unit, and accumulates coincidence information therefrom. Furthermore, the radiation tomography device 2 outputs to the image processing device 10 the list data in which the collected multiple pieces of coincidence information are arranged in time series.

[0025] The list data contains identification and detection time information of a pair of radiation detectors used in the coincidence detection of the photon pair. The list data may further contain information about the detection time difference (TOF information) of the pair of radiation detectors, information about the depth of photon interaction (DOI information) in the radiation detector, energy information of the photons detected by the radiation detectors, and the like.

[0026] The storage unit 14 stores the list data collected by the radiation tomography device 2. Furthermore, the storage unit 14 also stores a program for causing the reconstruction unit 11, the CNN processing unit 12, and the update unit 13 to perform corresponding processing steps. The reconstruction unit 11, the CNN processing unit 12, and the update unit 13 create the tomographic image of the object using the program and the list data stored in the storage unit 14.

[0027] The reconstruction unit 11 performs processing based on the iterative list-mode reconstruction method (see Non-Patent Document 2) to generate a first image. The processing based on the iterative list-mode approximate reconstruction method performed in the reconstruction unit 11 uses an iterative update formula such as LM-MLEM (Maximum Likelihood Expectation Maximization), LM-OSEM (Ordered Subset EM), LM-DRAMA (Dynamic Row Action Maximum Likelihood Algorithm), or the like.

[0028] The CNN processing unit 12 performs processing based on the DIP technique (see Non-Patent Document 1) to generate a second image. In the DIP-based processing performed in the CNN processing unit 12, input information is input to a CNN, the second image is generated by the CNN, and the CNN is further trained. The input information to be input to the CNN may be shape information of the object, an MRI image or a CT image of the object, or a random noise image.

[0029] The updating unit 13 updates a third image based on the first image and the second image. The details of the respective processes performed by the reconstruction unit 11, the CNN processing unit 12, and the updating unit 13 will be described later.

[0030] The storage unit 14 also stores the input information to be input to the CNN, as well as the first image, the second image, and the third image. In the image processing device 10, starting from a certain initial state, the respective processes of the reconstruction unit 11, the CNN processing unit 12, and the update unit 13 are repeatedly performed, and the tomographic image of the object is generated.

[0031] Fig. Figure 2 is a flowchart illustrating an image processing method. The image processing method includes a reconstruction step S1 performed by the reconstruction unit 11, a CNN processing step S2 performed by the CNN processing unit 12, and an update step S3 performed by the update unit 13. Starting from a certain initial state, the reconstruction step S1, the CNN processing step S2, and the update step S3 are performed multiple times (N times) to generate the tomographic image of the object.

[0032] In step S4, a value of a parameter n is set to an initial value of 0. In a subsequent step S5, the value of the parameter n is incremented by 1. After step S5, the reconstruction step S1, the CNN processing step S2, and the update step S3 are performed. In step S6 following these steps, the value of the parameter n is compared with N, and if it is determined that n is less than N, the process returns to step S5. If it is determined in step S6 that n has reached the value N, the repeated processing is terminated, and the tomographic image of the object is acquired.

[0033] In the following description, among the N-times repeated processing steps, an n-th reconstruction step S1 is referred to as a reconstruction step S1(n), an n-th CNN processing step S2 is referred to as a CNN processing step S2(n), and an n-th update step S3 is referred to as an update step S3(n). n is an integer of 1 or more and N or less.

[0034] The details of the processing of the image processing device 10 and the image processing method are described below. First, the list data U is formulated as shown in the following formula (1). t is a number indicating the coincidence event. T is the total number of events. i(t) is a number identifying a detector pair used in detecting the t-th event. [Formula 1] U={i(t)|t=1,2,⋯,T}

[0035] For the above-mentioned list data U, a constrained optimization problem is considered, which is represented by the following formula (2). x is the tomographic image. L(U|x) is a probability that indicates a probability that the list data U is observed from the tomographic image x. z is the input information to be input to the CNN. θ is a parameter that indicates a training state of the CNN, such as a connection weight and the like, and changes with the progress of the training of the CNN. f θ (z) is an image output of the CNN in the case where the input information z is input to the CNN with the training state θ. The constrained optimization problem of the above formula (2) is a problem of optimizing the tomographic image x and the CNN parameter θ such that the probability L(U|x) becomes high, under the constraint (x = f θ (z)) that the tomographic image x is the CNN output image fθ (z) will be. [Formula 2] maxL(U|x)stx=ƒθ(z)

[0036] As one aspect of a solution method for the optimization problem described above, there are a first aspect and a second aspect, as described below. The first aspect includes processing based on an alternating direction multiplier method (ADMM method). The second aspect includes processing based on a forward-backward splitting (FBS method). In addition, a De Pierro method is also included in the FBS method as a specific aspect of the FBS method. The following description describes the details of the image processing methods of the first and second aspects.

[0037] In the image processing method of the first aspect, the constrained optimization problem of the above formula (2) is rewritten based on an extended Lagrange function method and then solved using the ADMM method. In the extended Lagrange function method, the constraint in the above formula (2) is replaced with a regularization term, and the constrained optimization problem represented by the above formula (2) is rewritten as an unconstrained optimization problem represented by the following formula (3). ρ is a positive constant for adjusting a strength of regularization. µ is a parameter called a Lagrange multiplier or a dual variable, and is referred to as the "third image" in the following description. [Formula 3] min−L(U|x)+ρ2‖x−ƒθ(z)+μ‖2+ρ2‖μ‖2

[0038] In the first aspect, the ADMM solves the unconstrained optimization problem of the above formula (3) by repeatedly performing processes of the following formula (4) to formula (6). [Formula 4] x(n)=argmaxxL(U|x)−ρ2‖x−ƒθ(n−1)(z)+μ(n−1)‖2 [Formula 5] θ(n)=argminθ‖ƒθ(z)−x(n)−μ(n−1)‖2 [Formula 6] μ(n)=μ(n−1)+x(n)−ƒθ(n)(z)

[0039] Fig. Figure 3 is a diagram illustrating a sequence of the image processing method according to the first aspect. Before performing the repeated processing, the training state θ (0) of CNN, the first picture x (0) , the second picture f θ (0) (z) and the third image µ (0) are initialized. In addition, the second image f θ (0) (z) an image output by the CNN when the input information z is compared with the initial training state of θ (0)be input into the CNN.

[0040] In the n-th reconstruction step S1 (n), according to the above formula (4), the first image x (n) newly created by repeatedly performing the processing, an image that is created by updating the first image x (n-1) obtained based on the list data U, using the iterative list mode reconstruction method close to a difference (f θ (n-1) (z) - µ (n-1) ) between the second image f θ (n-1) (z) and the third image µ (n-1) bring to.

[0041] In the n-th CNN processing step S2(n), the input information z is input to the CNN according to the above formula (5), and the second image f θ (n) (z) is generated by the CNN, and further, the CNN is trained so that the generated second image f θ (n) (z) close to a sum (x (n) + µ (n-1)) of the first image x (n) and the third image µ (n-1) The training state of the CNN after the above training is set to θ (n) set.

[0042] In the n-th update step S3(n), according to the above formula (6), a difference (x (n) - f θ (n) (z)) between the first image x (n) and the second picture f θ (n) (z) to the third image µ (n-1) added and thus to the third image µ (n) updated.

[0043] When the N-times steps of repeated processing of reconstruction step S1, CNN processing step S2 and update step S3 are completed, either the first image x (N) or the second picture f θ (N) (z) obtained as a result of repeated processing is set as the tomographic image of the object.

[0044] In the first aspect, the first image x and the second image f θ (z) is not optimized simultaneously, but the first image x and the second image f θ (z) are alternately optimized, and thus the problem is easy to solve. Furthermore, the processing of the reconstruction step S1 performed by the reconstruction unit 11 and the processing of the CNN processing step S2 performed by the CNN processing unit 12 can be performed using conventional techniques, such as those described in Non-Patent Documents 1 and 2, and is therefore easy to implement.

[0045] Fig. 4 to Fig. 11 are diagrams showing the results of a simulation performed to confirm the effect of the image processing method according to the first aspect. In the present simulation, simulation data was generated by an MC simulation of a head PET device using a digital brain phantom image, and the effect of the image processing method of the first aspect was confirmed using the generated simulation data.

[0046] Fig. 4 and Fig. 5 contain diagrams, each showing a phantom image (correct image). Each of the figures (a) in Fig. 4 and (b) in Fig. 4 shows an image of the transverse section, (a) in Fig. 5 shows an image of the coronal section, and (b) in Fig. Figure 5 shows an image of the sagittal section.

[0047] Fig. 6 and Fig. 7 contain diagrams each showing the tomographic image obtained by the image processing method according to a first comparative example. In this case, both (a) in Fig. 6 and (b) in Fig. 6 a picture of the transverse section, (a) in Fig. 7 shows an image of the coronal section and (b) in Fig. Figure 7 shows a sagittal section image. The tomographic image of the first comparative example is obtained by reconstruction using only the iterative update formula of the LM-DRAMA of the iterative list-mode reconstruction method, and noise reduction is not performed.

[0048] Fig. 8 and Fig. 9 are diagrams each showing the tomographic image obtained by the image processing method according to a second comparative example. (a) Fig. 8 and (b) in Fig. 8 each shows an image of the transverse section, (a) in Fig. 9 shows a picture of the coronal section, and (b) in Fig. Figure 9 shows a sagittal section image. The tomographic image of the second comparative example is obtained by reducing the noise using the DIP technique for the reconstructed tomographic image of the first comparative example. The parameter update frequency of the CNN is set to 20. The input information supplied to the CNN is set to the MRI image.

[0049] Fig. 10 and Fig. 11 are diagrams each showing the tomographic image obtained by the image processing method according to an example. (a) Fig. 10 and (b) in Fig. 10 each shows an image of the transverse section, (a) in Fig. 11 shows an image of the coronal section, and (b) in Fig. Figure 11 shows a sagittal section image. The tomographic image of the example is created using the image processing method according to the first aspect. It is set to ρ = 0.05. The MRI image is set as the input information z for the CNN. The number of iterations in reconstruction step S1 is set to 2, the number of iterations of CNN training in CNN processing step S2 is set to 20, and the total number of iterations N is set to 200.

[0050] As can be seen from the comparison of the above tomographic images, in the tomographic image obtained by the image processing method according to the example, the noise is reduced, and also the structure of the cerebral cortex is well reconstructed.

[0051] Fig. Figure 12 is a graph showing a PSNR of the tomographic image obtained by each of the first and second comparative examples and the example. The PSNR is a peak signal-to-noise ratio (unit: dB), which is an index of noise. Fig. Figure 13 is a graph showing a CRC of the tomographic image obtained by each of the first and second comparative examples and the example. The CRC is a contrast recovery coefficient of a tumor, which is an index of the quantity.

[0052] Compared with the tomographic images of the first and second comparative examples, the tomographic image of the example exhibits large PSNR and CRC values, respectively, and the CRC is close to the ideal value of 1.0. As can be seen from the above results, the image processing method according to the example makes it possible to generate a highly quantitative tomographic image while suppressing an increase in noise artifacts.

[0053] Next, the image processing method according to the second aspect will be described. In the image processing method of the second aspect, the constrained optimization problem of the above formula (2) is solved by the FBS method based on a maximum a posteriori estimation (MAP) framework. In the MAP estimation method, the constrained optimization problem represented by the above formula (2) is rewritten into an unconstrained optimization problem represented by the following formula (7). [Formula 7] min−L(U|x)+ρ2‖x−ƒθ(z)‖2

[0054] In the second aspect, the FBS solves the unconstrained optimization problem of the above formula (7) by repeatedly performing processes of the following formula (8) to formula (11). [Formula 8] xML(n)=argmaxxL(U|x) [Formula 9] θ(n)=argminθ‖ƒθ(z)−x(n−1)‖2 [Formula 10] xReg(n)=x(n−1)+ργ(ƒθ(n)(z)−x(n−1)) [Formula 11] x(n)=2xML(n)(1−1γωxReg(n))+(1−1γωxReg(n))2+41γωxML(n)

[0055] In this case, γ is a predefined parameter image. ω is an image representing the sensitivity of the detector for each pixel. If the probability that the γ-ray pair emitted by pixel j is detected by detector pair i is set to p ij is fixed, the sensitivity image w is represented by the following formula (12). [Formula 12] ωj=∑ipij

[0056] Fig. Fig. 14 is a diagram showing a sequence of the image processing method according to the second aspect. Before performing the repeated processing, the training state θ (0) of CNN and the third image x (0) initialized in each case.

[0057] In the n-th reconstruction step S1(n), according to the above formula (8), the first image xML (n) generated by updating the third image x (n-1) based on the list data U using the iterative list mode reconstruction method.

[0058] In the n-th CNN processing step S2(n), the input information z is input to the CNN according to the above formula (9), and the second image f θ (n) (z) is generated by the CNN, and further, the CNN is trained so that the generated second image f θ (n) (z) close to the third image x (n-1) The training state of the CNN after the above training is set to θ (n) set.

[0059] In the n-th update step S3(n), according to the above formula (10) and formula (11), the third image x (n-1) based on the first image x ML (n) and the second image f θ (n) (z) to the third image x (n) updated.

[0060] When the N-times steps of repeated processing of reconstruction step S1, CNN processing step S2 and update step S3 are completed, the first image x ML (N) , the second picture f θ (N) (z) or the third image x (N) obtained as a result of repeated processing are defined as a tomographic image of the object.

[0061] In the second aspect, the respective processes of the reconstruction step S1(n) and the CNN processing step S2(n) may be performed in any order or in parallel.

[0062] The De Pierro method corresponds to a method where ρ = 1 and γ = 1 / ω are set in the FBS method. That is, the De Pierro method uses the following formulas (13) and (14) instead of the above formulas (10) and (11).

[0063] The processing content of each reconstruction step S1, CNN processing step S2, and update step S3 is the same as described above. [Formula 13] xReg(n)=x(n−1)+1ω(ƒθ(n)(z)−x(n−1)) [Formula 14] x(n)=2xML(n)(1−xReg(n))+(1−xReg(n))2+4xML(n)

[0064] In the second aspect, the first image x ML and the second picture f θ (z) is not optimized at the same time, but the first image x ML and the second picture f θ(z) are optimized separately, making the problem easy to solve. Furthermore, the processing of the reconstruction step S1 performed by the reconstruction unit 11 and the processing of the CNN processing step S2 performed by the CNN processing unit 12 can be performed using conventional techniques, such as those described in Non-Patent Documents 1 and 2, and is therefore easy to implement.

[0065] As in the case of the first aspect, noise is reduced in the tomographic image obtained by the image processing method according to the second aspect, and the structure of the cerebral cortex is also well reconstructed. In the second aspect, it is also possible to generate a highly quantitative tomographic image while simultaneously suppressing an increase in noise artifacts.

[0066] The present invention is not limited to the above-described embodiments and configuration examples, and various modifications are possible. For example, the radiation tomography device 2 is a PET device in the above embodiment, and further, it may be a SPECT device.

[0067] The image processing apparatus of the first aspect according to the above embodiment is an image processing apparatus for generating a tomographic image based on list data collected by a radiation tomography apparatus, and includes (1) a reconstruction unit for regenerating a first image by repeatedly performing processing to bring an image obtained by updating the first image based on the list data close to a difference between a second image and a third image using an iterative list-mode reconstruction method; (2) a CNN processing unit for inputting input information to a convolutional neural network, generating the second image by the convolutional neural network, and training the convolutional neural network.to bring the generated second image close to a sum of the first image and the third image, and (3) an updating unit for updating the third image based on the first image and the second image, wherein, starting from an initial state of each training state of the convolutional neural network, the first image, the second image, and the third image, the generation of the first image by the reconstruction unit, the generation of the second image and the training of the convolutional neural network by the CNN processing unit, and the updating of the third image by the updating unit are repeatedly performed, and the first image or the second image obtained by the repeated processing is set as the tomographic image.

[0068] The image processing apparatus of the second aspect according to the above embodiment is an image processing apparatus for generating a tomographic image based on list data collected by a radiation tomography apparatus, and includes (1) a reconstruction unit for generating a first image by updating a third image based on the list data using an iterative list-mode reconstruction method; (2) a CNN processing unit for inputting input information to a convolutional neural network, generating a second image by the convolutional neural network, and training the convolutional neural network to bring the generated second image close to the third image; and (3) an updating unit for updating the third image based on the first image and the second image.wherein, starting from an initial state of each training state of the convolutional neural network and the third image, the generation of the first image by the reconstruction unit, the generation of the second image and the training of the convolutional neural network by the CNN processing unit, and the updating of the third image by the updating unit are repeatedly performed, and any one of the first image, the second image, and the third image obtained by repeated processing is set as the tomographic image.

[0069] In the above image processing apparatus of the first or second aspect, the CNN processing unit may input an image representing shape information of an object to the convolutional neural network as input information.

[0070] In the above image processing device of the first or second aspect, the CNN processing unit may input an MRI image of an object to the convolutional neural network as input information. Further, the CNN processing unit may input a CT image of an object to the convolutional neural network as input information.

[0071] In the above image processing apparatus of the first or second aspect, the CNN processing unit may input a random noise image as input information to the convolutional neural network.

[0072] The radiation tomography system according to the above embodiment includes a radiation tomography apparatus for collecting list data for reconstructing a tomographic image of an object; and the image processing device of the above configuration for generating the tomographic image based on the list data collected by the radiation tomography apparatus.

[0073] The image processing method of the first aspect according to the above embodiment is an image processing method for generating a tomographic image based on list data collected by a radiation tomography apparatus, and includes (1) a reconstruction step of newly generating a first image by repeatedly performing processing to bring an image obtained by updating the first image based on the list data using an iterative list-mode reconstruction method close to a difference between a second image and a third image; (2) a CNN processing step of inputting input information to a convolutional neural network, generating the second image by the convolutional neural network, and training the convolutional neural network to bring the generated second image close to a sum of the first image and the third image;and (3) an updating step of updating the third image based on the first image and the second image, wherein, starting from an initial state of each training state of the convolutional neural network, the first image, the second image, and the third image, the generation of the first image in the reconstruction step, the generation of the second image and the training of the convolutional neural network in the CNN processing step, and the updating of the third image in the updating step are repeatedly performed, and the first image or the second image obtained by the repeated processing is set as the tomographic image.

[0074] The image processing method of the second aspect according to the above embodiment is an image processing method for generating a tomographic image based on list data collected by a radiation tomography apparatus, and includes (1) a reconstruction step of generating a first image by updating a third image based on the list data using an iterative list-mode reconstruction method; (2) a CNN processing step of inputting input information to a convolutional neural network, generating a second image by the convolutional neural network, and training the convolutional neural network to bring the generated second image close to the third image; and (3) an updating step of updating the third image based on the first image and the second image.and starting from an initial state of each training state of the convolutional neural network and the third image, the generation of the first image in the reconstruction step, the generation of the second image and the training of the convolutional neural network in the CNN processing step, and the updating of the third image in the updating step are repeatedly performed, and any one of the first image, the second image, and the third image obtained by repeated processing is set as the tomographic image.

[0075] In the above image processing method of the first or second aspect, in the CNN processing step, an image representing shape information of an object may be input to the convolutional neural network as input information.

[0076] In the above image processing method of the first or second aspect, in the CNN processing step, an MRI image of an object may be input to the convolutional neural network as input information. Further, in the CNN processing step, a CT image of an object may be input to the convolutional neural network as input information.

[0077] In the above image processing method of the first or second aspect, in the CNN processing step, a random noise image may be input to the convolutional neural network as input information. Industrial applicability

[0078] The present invention can be used as an image processing apparatus and an image processing method capable of generating a tomographic image in which noise is reduced based on list data collected by a radiation tomography apparatus. List of reference symbols

[0079] 1 - radiation tomography system, 2 - radiation tomography device, 10 - image processing device, 11 - reconstruction unit, 12 - CNN processing unit, 13 - updating unit, 14 - storage unit. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited non-patent literature

[0000] Kuang Gong et al. “PET Image Reconstruction Using Deep Image Prior,” IEEE Transactions on Medical Imaging, December 2018

[0011] A. J. Reader et al., „One-Pass List-Mode EM Algorithm for High-Resolution 3-D PET Image Reconstruction Into Large Arrays“, IEEE Transactions on Nuclear Science, Vol.49, No.3, pp.693-699, 2002

[0011]

Claims

[1] An image processing apparatus for generating a tomographic image based on list data collected by a radiation tomography apparatus, the image processing apparatus comprising: a reconstruction unit for regenerating a first image by repeatedly performing processing of bringing an image obtained by updating the first image based on the list data closer to a difference between a second image and a third image using an iterative list mode reconstruction method; a CNN processing unit for inputting input information to a convolutional neural network, generating the second image by the convolutional neural network, and training the convolutional neural network to bring the generated second image close to a sum of the first image and the third image; and an updating unit for updating the third image based on the first image and the second image, wherein starting from an initial state of each training state of the neural convolutional network, the first image, the second image and of the third image, the generation of the first image by the reconstruction unit, the generation of the second image and the training of the convolutional neural network by the CNN processing unit, and the updating of the third image by the updating unit are repeatedly performed, and one of the first image or the second image obtained by the repeated processing is set as the tomographic image. [2] An image processing apparatus for generating a tomographic image based on list data collected by a radiation tomography apparatus, the image processing apparatus comprising: a reconstruction unit for generating a first image by updating a third image based on the list data using an iterative list mode reconstruction method; a CNN processing unit for inputting input information to a convolutional neural network, generating a second image by the convolutional neural network, and training the convolutional neural network to bring the generated second image close to the third image; and an updating unit for updating the third image based on the first image and the second image, wherein starting from an initial state of each training state of the convolutional neural network and the third image, the generation of the first image by the reconstruction unit, the generation of the second image and the training of the convolutional neural network by the CNN processing unit, and the updating of the third image by the updating unit are repeatedly performed, and any one of the first image, the second image, and the third image obtained by repeated processing is set as the tomographic image. [3] The image processing apparatus according to claim 1 or 2, wherein the CNN processing unit inputs an image representing shape information of an object as input information to the convolutional neural network. [4] The image processing apparatus according to claim 1 or 2, wherein the CNN processing unit inputs an MRI image of an object as input information to the convolutional neural network. [5] The image processing apparatus according to claim 1 or 2, wherein the CNN processing unit inputs a CT image of an object as input information to the convolutional neural network. [6] The image processing apparatus according to claim 1 or 2, wherein the CNN processing unit inputs a random noise image as input information to the convolutional neural network. [7] A radiation tomography system comprising: a radiation tomography device for collecting list data for reconstructing a tomographic image of an object; and the image processing apparatus according to any one of claims 1 to 6 for generating the tomographic image based on the list data collected by the radiation tomography apparatus. [8] An image processing method for generating a tomographic image based on list data collected by a radiation tomography apparatus, the image processing method comprising: a reconstruction step of regenerating a first image by repeatedly performing the processing of bringing an image, obtained by updating the first image based on the list data using an iterative list mode reconstruction method close to a difference between a second image and a third image; a CNN processing step of inputting input information to a convolutional neural network, generating the second image by the convolutional neural network, and training the convolutional neural network to bring the generated second image close to a sum of the first image and the third image; and an updating step for updating the third image based on the first image and the second image, wherein starting from an initial state of each training state of the neural convolutional network, the first image, the second image and the third image, the generation of the first image in the reconstruction step, the generation of the second image and the training of the convolutional neural network in the CNN processing step and the updating of the third image is carried out repeatedly in the updating step and one of the first image or the second image, which is obtained by repeated processing, as the tomographic image is determined. [9] An image processing method for generating a tomographic image based on list data collected by a radiation tomography apparatus, the image processing method comprising: a reconstruction step of generating a first image by updating a third image based on the list data using an iterative list mode reconstruction method; a CNN processing step of inputting input information to a convolutional neural network, generating a second image by the convolutional neural network, and training the convolutional neural network to bring the generated second image close to the third image; and an updating step for updating the third image based on the first image and the second image, wherein starting from an initial state of each training state of the convolutional neural network and the third image, the generation of the first image in the reconstruction step, the generation of the second image and the training of the convolutional neural network in the CNN processing step, and the updating of the third image in the updating step are repeatedly performed, and any one of the first image, the second image, and the third image obtained by repeated processing is set as the tomographic image. [10] The image processing method according to claim 8 or 9, wherein in the CNN processing step, an image representing shape information of an object is input to the convolutional neural network as input information. [11] The image processing method according to claim 8 or 9, wherein in the CNN processing step, an MRI image of an object is input to the convolutional neural network as input information. [12] The image processing method according to claim 8 or 9, wherein in the CNN processing step, a CT image of an object is input to the convolutional neural network as input information. [13] The image processing method according to claim 8 or 9, wherein in the CNN processing step, a random noise image is input to the convolutional neural network as input information.