Image processing device and image processing method

The proposed CNN architecture with an encoder-decoder structure and projection combining unit addresses the data requirement issue, improving tomographic image creation performance by enabling effective learning from reduced data sets.

JP2024157161APending Publication Date: 2024-11-07HAMAMATSU PHOTONICS KK
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2023071328
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-25
Publication Date
2024-11-07

AI Technical Summary

Technical Problem

Existing methods, such as those using convolutional neural networks (CNN), require a large amount of data for learning to create tomographic images from measurement data due to the heterogeneity between input sinograms and output images in different spaces, leading to poor performance when insufficient data is available.

Method used

A convolutional neural network architecture with an encoder-decoder structure and a projection combining unit that performs projections from measurement space to real space, allowing the network to learn directly from measurement images without relying solely on memorization of training data.

Benefits of technology

This approach reduces the required data volume for CNN learning and enhances the performance of creating tomographic images, especially when dealing with unknown data, by enabling the network to generalize better and develop algorithms for image reconstruction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024157161000001_ABST
    Figure 2024157161000001_ABST
Patent Text Reader

Abstract

To provide an image processing device that can prevent an increase in amount of data required for learning of CNN and improve performance of tomographic image creation by CNN.SOLUTION: The image processing device creates a measurement image on a measurement space different from a real space based on measurement data of a subject acquired by measurement performed by a tomographic device, inputs the measurement image to a CNN, and creates a tomographic image on the real space by means of the CNN. The CNN is based on a U-net structure including an encoder unit (21), a decoder unit (22), and a bottleneck unit (23), and further includes a projection coupling unit (24). The projection coupling unit (24) subjects any of feature maps in the encoder unit (21) to projection processing from a sinogram space (measurement space) to the real space, and couples the feature map after the projection processing to a feature map of the same size in the decoder unit (22).SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an apparatus and method for generating a tomographic image of a subject based on measurement data of the subject acquired by measurement using a tomography apparatus. [Background technology]

[0002] Examples of tomography devices that acquire measurement data for creating a tomographic image of a subject include a PET (Positron Emission Tomography) device, a SPECT (Single Photon Emission Computed Tomography) device, an X-ray CT (Computed Tomography) device, and an MRI (Magnetic Resonance Imaging) device. The measurement data of a subject acquired by these tomography devices does not directly represent a tomographic image of the subject in real space. To create a tomographic image of a subject in real space based on the measurement data of the subject acquired by a tomography device, first, a measurement image in a measurement space different from the real space is created based on the measurement data, and then a tomographic image in real space is created based on the measurement image in the measurement space. The health condition of the subject can be diagnosed based on this tomographic image.

[0003] In the case of PET, SPECT and X-ray CT devices, the measurement space is a sinogram space, and the measurement image is a sinogram. In the case of a 3D PET device, the sinogram is expressed as a histogram showing the frequency of acquiring coincidence information (frequency of occurrence of coincidence events) in a sinogram space represented by four variables r, θ, z and δ. The variable r represents the radial position of the coincidence line (the line connecting the two detectors that simultaneously counted photon pairs). The variable θ represents the azimuth angle of the coincidence line. The variable z represents the central axial position of the midpoint of the coincidence line. The variable δ represents the polar angle of the coincidence line. In the case of an MRI device, the measurement space is k-space represented by a variable of spatial frequency, and the measurement image is composed of k-space data.

[0004] Image reconstruction processing is performed when creating a tomographic image in real space based on a measurement image in measurement space. Several methods are known as image reconstruction methods for converting a sinogram into a tomographic image. For example, ML-EM (Maximum Likelihood Expectation Maximization), OSEM (Ordered Subset Expectation Maximization), RAMLA (Row Action Maximum Likelihood Algorithm), and DRAMA (Dynamic RAMLA) are known. These image reconstruction methods alternately repeat forward projection from real space to sinogram space and back projection from sinogram space to real space to reconstruct a tomographic image in an iterative manner.

[0005] Also, a technology has been proposed that uses a neural network to create a tomographic image in real space from a measurement image in a measurement space. This technology is expected to be able to create tomographic images with higher performance. The technology described in Non-Patent Document 1 aims to input a sinogram to a Convolutional Neural Network (CNN) and create a tomographic image using the CNN. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] I. Haggstrom et al. “DeepPET: Adeep encoder-decoder network for directly solving the PET image reconstructioninverse problem,” Med. Image Anal. Vol.54, pp.253-262, 2019. Summary of the Invention [Problem to be solved by the invention]

[0007] The technology described in Non-Patent Document 1 requires that CNN be trained in advance using pairs of sinograms and tomographic images of many subjects. The input image to CNN is a sinogram in sinogram space, while the output image from CNN is a tomographic image in real space. The input image (sinogram) and the output image (tomographic image) are heterogeneous images in different spaces, and look very different. Therefore, creating a tomographic image based on a sinogram by CNN is a difficult task. For this reason, the technology described in Non-Patent Document 1 requires a large amount of data (pairs of sinograms and tomographic images) for training CNN.

[0008] If the amount of data used for learning is small, the CNN does not learn an algorithm for creating a tomographic image from a sinogram, but merely stores the pairs of sinograms and tomographic images used for learning. When an unknown sinogram is input, the CNN outputs a tomographic image corresponding to the sinogram that is closest to the input sinogram among the sinograms used for learning.

[0009] Thus, in the technology described in Non-Patent Document 1, if the amount of data used for training the CNN is small, the performance of creating a tomographic image by the CNN is low. In order to improve the performance of creating a tomographic image by the CNN, a large amount of data is required for training the CNN.

[0010] The problems in creating cross-sectional images using CNN as described above exist not only when creating cross-sectional images from sinograms in a PET device, but also when creating cross-sectional images in real space from measurement images in measurement space in a SPECT device, an X-ray CT device, and an MRI device.

[0011] The present invention has been made to solve the above problems, and has an object to provide an image processing device and an image processing method that can suppress an increase in the amount of data required for CNN training and improve the performance of creating tomographic images by CNN. [Means for solving the problem]

[0012] The image processing apparatus of the present invention is an apparatus that creates a tomographic image of a subject in real space based on measurement data of the subject acquired by measurement using a tomography apparatus.

[0013] A first aspect of the image processing device of the present invention includes (1) a measurement image creation unit that creates a measurement image in a measurement space different from the real space based on measurement data, and (2) a CNN processing unit that inputs the measurement image to a convolutional neural network and creates a tomographic image by the convolutional neural network. The convolutional neural network includes (a) an encoder unit that inputs the measurement image and updates a feature map by convolutional calculations by each of multiple convolutional layers, gradually reducing the spatial size of the feature map, (b) a decoder unit that is provided after the encoder unit and updates the feature map by convolutional calculations by each of multiple convolutional layers, gradually expanding the spatial size of the feature map, and (c) a projection and combination unit that performs a projection process from the measurement space to the real space on any of the feature maps in the encoder unit and combines the feature map after the projection process with a feature map of the same size in the decoder unit.

[0014] In a second aspect of the image processing device of the present invention, in addition to the first aspect, a CNN training unit that trains a convolutional neural network using measurement images and tomographic images of each of a plurality of subjects is further provided.

[0015] In a third aspect of the image processing device of the present invention, in addition to the first or second aspect, the CNN processing unit performs a projection process from the measurement space to the real space on the measurement image input to the encoder unit, and adds the image after the projection process to an output image from the decoder unit.

[0016] In a fourth aspect of the image processing device of the present invention, in addition to any of the first to third aspects, the projection combination unit performs a projection process from the measurement space to the real space on a feature map of a maximum size in the encoder unit, and combines the feature map after the projection process with a feature map of the same size in the decoder unit.

[0017] In a fifth aspect of the image processing device of the present invention, in addition to any of the first to third aspects, the projection combination unit performs a projection process from the measurement space to the real space on feature maps of all sizes in the encoder unit, and combines the feature maps after the projection process with feature maps of the same size in the decoder unit.

[0018] A tomography system according to the present invention includes a tomography device that acquires measurement data for creating a tomography image of a subject, and the image processing device according to the present invention that creates a tomography image based on the measurement data.

[0019] The image processing method of the present invention is a method for creating a tomographic image of a subject in real space based on measurement data of the subject acquired by measurement using a tomography apparatus.

[0020] A first aspect of the image processing method of the present invention includes (1) a measurement image creation step of creating a measurement image in a measurement space different from the real space based on measurement data, and (2) a CNN processing step of inputting the measurement image to a convolutional neural network and creating a tomographic image by the convolutional neural network. The convolutional neural network includes (a) an encoder unit that inputs the measurement image and updates a feature map by convolutional calculations in each of a plurality of convolutional layers, gradually reducing the spatial size of the feature map, (b) a decoder unit that is provided after the encoder unit and updates the feature map by convolutional calculations in each of a plurality of convolutional layers, gradually expanding the spatial size of the feature map, and (c) a projection and combination unit that performs a projection process from the measurement space to the real space on any of the feature maps in the encoder unit and combines the feature map after the projection process with a feature map of the same size in the decoder unit.

[0021] A second aspect of the image processing method of the present invention includes, in addition to the first aspect, a CNN learning step of training a convolutional neural network using measurement images and tomographic images of each of a plurality of subjects.

[0022] In a third aspect of the image processing method of the present invention, in addition to the first or second aspect, the CNN processing step performs a projection process from the measurement space to the real space on the measurement image input to the encoder unit, and adds the image after the projection process to an output image from the decoder unit.

[0023] In a fourth aspect of the image processing method of the present invention, in addition to any of the first to third aspects, the projection combination unit performs a projection process from the measurement space to the real space on a feature map of a maximum size in the encoder unit, and combines the feature map after the projection process with a feature map of the same size in the decoder unit.

[0024] In a fifth aspect of the image processing method of the present invention, in addition to any of the first to third aspects, the projection combination unit performs a projection process from the measurement space to the real space on feature maps of all sizes in the encoder unit, and combines the feature maps after the projection process with feature maps of the same size in the decoder unit. Effect of the Invention

[0025] According to the present invention, it is possible to suppress an increase in the amount of data required for CNN training and improve the performance of creating tomographic images by CNN. [Brief description of the drawings]

[0026] [Figure 1] FIG. 1 is a diagram showing the configuration of a tomography system 1. As shown in FIG. [Diagram 2] FIG. 2 is a diagram illustrating an example of the configuration of a CNN. [Diagram 3] FIG. 3 is a diagram for explaining a projection process between a sinogram space (measurement space) and a real space. [Figure 4]FIG. 4 is a diagram showing another example of the configuration of a CNN. [Diagram 5] FIG. 5 is a diagram showing a tomographic image obtained when noise-free sample 1 is used in the comparative example. [Figure 6] FIG. 6 is a diagram showing a tomographic image obtained when a noise-free sample 19 is used in the comparative example. [Figure 7] FIG. 7 is a diagram showing a tomographic image obtained when noise-free sample 1 is used in the example. [Figure 8] FIG. 8 is a diagram showing a tomographic image obtained when a noise-free sample 19 is used in the example. [Figure 9] FIG. 9 is a diagram showing a tomographic image obtained when Sample 1 having noise is used in the comparative example. [Figure 10] FIG. 10 is a diagram showing a tomographic image obtained when a sample 19 having noise is used in the comparative example. [Figure 11] FIG. 11 is a diagram showing a tomographic image obtained when Sample 1 having noise is used in the example. [Figure 12] FIG. 12 is a diagram showing a tomographic image obtained when a sample 19 having noise is used in the example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] Hereinafter, the embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are given the same reference numerals, and duplicated descriptions are omitted. The present invention is not limited to these examples, but is indicated by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.

[0028] 1 is a diagram showing the configuration of a tomography system 1. The tomography system 1 is a system for creating a tomography image of a subject, and includes a tomography device 2 and an image processing device 10. The tomography device 2 acquires measurement data for creating a tomography image of the subject, and is, for example, a PET device, a SPECT device, an X-ray CT device, or an MRI device. The image processing device 10 includes a storage unit 11, a measurement image creation unit 12, a correction unit 13, a CNN processing unit 14, and a CNN learning unit 15. The image processing method includes a storage step, a measurement image creation step, a correction step, a CNN processing step, and a CNN learning step.

[0029] The image processing device 10 creates a tomographic image of the subject in real space based on measurement data of the subject acquired by measurement using the tomography device 2. The image processing device 10 includes an arithmetic processing unit including a CPU and GPU that perform various processes, a storage unit including a hard disk drive and RAM that store programs and data for performing the processes, a display unit such as a liquid crystal display that displays processing conditions and processing results, and an input unit including a keyboard, mouse, etc. that accepts input of processing conditions, etc. The image processing device 10 may be configured by a computer.

[0030] The image processing device 10 includes a memory unit 11 that stores programs and data, as well as a measurement image creation unit 12, a correction unit 13, a CNN processing unit 14, and a CNN learning unit 15 as calculation processing units that perform various processes. In a storage step, the memory unit 11 inputs measurement data of the subject acquired by measurement using the tomography device 2, and stores the measurement data. In a measurement image creation step, the measurement image creation unit 12 creates a measurement image in a measurement space different from the real space based on the measurement data stored in the memory unit 11. In a correction step, the correction unit 13 performs various corrections (e.g., sensitivity correction, absorption correction, scattering correction) on the measurement image created by the measurement image creation unit 12.

[0031] In the CNN processing step, the CNN processing unit 14 inputs the measurement image in the measurement space after being corrected by the correction unit 13 to a convolutional neural network (CNN), and creates a tomographic image in the real space by this CNN. In the CNN learning step, the CNN learning unit 15 trains the CNN using the measurement images and tomographic images of each of the multiple subjects. The image processing device 10 after CNN learning may not be equipped with the CNN learning unit 15.

[0032] When the tomography apparatus 2 is a PET apparatus, a SPECT apparatus or an X-ray CT apparatus, the measurement space is a sinogram space and the measurement image is a sinogram. When the tomography apparatus 2 is an MRI apparatus, the measurement space is a k-space and the measurement image is composed of k-space data.

[0033] In the following description, the tomography device 2 is assumed to be a PET device. The PET device is a radiation tomography device capable of acquiring a tomographic image of a subject (living body). The PET device is provided with a detection unit having a large number of small radiation detectors arranged around a measurement space in which the subject is placed.

[0034] The PET device detects gamma-ray photon pairs with energy of 511 keV that are generated in association with the annihilation of electrons and positrons in a subject to which a positron-emitting isotope (RI source) has been administered, by a coincidence method using a detection unit, and collects this coincidence information. This coincidence information includes position information of each of the two radiation detectors that have counted the photon pairs in coincidence, and the coincidence line (the line connecting the two radiation detectors that have counted the photon pairs in coincidence) can be obtained from this position information. The memory unit 11 stores the coincidence information collected by the PET device.

[0035] The measurement image creation unit 12 creates a sinogram based on the collected coincidence information stored in the memory unit 11. The sinogram (measurement image) is expressed as a histogram showing the frequency of acquiring coincidence information (occurrence frequency of coincidence events) in a sinogram space (measurement space) in which the position and orientation of the coincidence line are variables. The CNN processing unit 14 inputs the sinogram (measurement image) in the sinogram space (measurement space) to a CNN, and creates a tomographic image in real space using this CNN.

[0036] FIG. 2 is a diagram showing an example of the configuration of a CNN. The CNN shown in this figure is based on a U-net structure including an encoder unit 21, a decoder unit 22, and a bottleneck unit 23, and further includes a projection combination unit 24. In this figure, feature maps after processing of a convolution layer (Conv3x3, stride 1), a normalization layer (Batch Normalization), a nonlinear activation function (Leaky ReLU), downsampling (Conv 4x4, stride 2), and upsampling are shown in different grayscales. In addition, this figure also shows the size of each feature map, with the number of pixels of each of a sinogram 31 input to the CNN and a tomographic image 32 output from the CNN being 128×128.

[0037] The convolution layer moves the filter relative to the previously created feature map (or input image) and performs a convolution operation between the feature map and the filter at each position to create a new feature map of the same spatial size. The normalization layer normalizes the features for each mini-batch of feature maps. The nonlinear activation function inputs the weighted sum z of each value of the feature map into the activation function σ(z)=max{0,z} and outputs the function value σ(z). Downsampling performs a convolution operation between the feature map and the filter to reduce the spatial size of the feature map to 1 / 2×1 / 2 and double the number of channels of the feature map. Upsampling expands the spatial size of the feature map by 2×2 times through linear interpolation and performs a convolution operation between the feature map and the filter to reduce the number of channels of the feature map to 1 / 2.

[0038] The encoder unit 21 has a convolution layer, a normalization layer, a nonlinear activation function, and downsampling. The encoder unit 21 inputs a sinogram (measurement image) in a sinogram space (measurement space) and updates a feature map by convolution, a normalization layer, and a nonlinear activation function. After performing processing by the convolution, normalization layer, and nonlinear activation function several times, the encoder unit 21 reduces the spatial size of the feature map to 1 / 2×1 / 2 by downsampling and doubles the number of channels of the feature map. In this figure, the spatial size of the feature map is initially 128×128, but is reduced to 64×64, 32×32, 16×16, and so on.

[0039] The decoder unit 22 has a convolution layer, a normalization layer, a nonlinear activation function, and upsampling. The decoder unit 22 is provided after the encoder unit 21, and by upsampling, the decoder unit 22 doubles the spatial size of the feature map and reduces the number of channels of the feature map to 1 / 2×1 / 2. In this figure, the spatial size of the feature map is expanded to 16×16, 32×32, 64×64, and 128×128. After performing the upsampling process, the decoder unit 22 updates the feature map using the convolution, normalization layer, and nonlinear activation function.

[0040] The bottleneck unit 23 is provided between the encoder unit 21 and the decoder unit 22. The bottleneck unit 23 has a convolution layer, a normalization layer, and a nonlinear activation function, and updates the feature map using these. The bottleneck unit 23 doubles the number of channels while maintaining the spatial size of the feature map.

[0041] The encoder unit 21 inputs a sinogram (measurement image) in a sinogram space (measurement space) and extracts the features of the input sinogram as a feature map. Meanwhile, the decoder unit 22 gradually reconstructs a tomographic image in real space based on the feature map indicating the extracted sinogram features.

[0042] The projection combining unit 24 performs a projection process from a sinogram space (measurement space) to a real space on any of the feature maps in the encoder unit 21, and combines the feature map after the projection process with a feature map of the same size in the decoder unit 22. The projection combining unit 24 may perform a projection process on any of the feature maps in the encoder unit 21, but it is preferable to perform a projection process on the feature map of the largest size in the encoder unit 21, and it is more preferable to perform a projection process on feature maps of all sizes in the encoder unit 21. In either case, the spatial size of the feature map is not changed by the projection process, and the feature map after the projection process is combined with a feature map of the same spatial size in the decoder unit 22. The combination of the feature maps after the projection process with the feature maps in the decoder unit may be a combination of each channel, or a combination of channels. A feature map of a spatial size that is not projection combined may be combined (skip combined) as it is with a feature map of the same spatial size in the decoder unit 22.

[0043] In FIG. 2, the projection and combination unit 24 performs a projection process on feature maps of all sizes in the encoder unit 21, and combines the feature maps after the projection process with feature maps of the same sizes in the decoder unit 22. That is, the projection and combination unit 24 projects a feature map 41 of 128×128 size in the encoder unit 21, and combines a feature map 51 after the projection process with a feature map of 128×128 size in the decoder unit 22. The projection and combination unit 24 projects a feature map 42 of 64×64 size in the encoder unit 21, and combines a feature map 52 after the projection process with a feature map of 64×64 size in the decoder unit 22. The projection and combination unit 24 projects a feature map 43 of 32×32 size in the encoder unit 21, and combines a feature map 53 after the projection process with a feature map of 32×32 size in the decoder unit 22. The projection and combination unit 24 projects a feature map 44 of 16×16 size in the encoder unit 21, and combines a feature map 54 after the projection process with a feature map of 16×16 size in the decoder unit 22.

[0044] FIG. 3 is a diagram for explaining the projection process between sinogram space (measurement space) and real space. This diagram shows the relationship between a sinogram 31 and a tomographic image 32 in a 2D PET device. In the iterative image reconstruction method, the projection from real space to sinogram space is called forward projection, and the projection from sinogram space to real space is called back projection. In the iterative image reconstruction method, forward projection and back projection are alternately repeated to reconstruct a tomographic image in an iterative manner.

[0045] Forward projection is a process of converting a tomographic image 32 in real space into a sinogram 31, and is expressed by the following (1). p is a sinogram, and f is a tomographic image. A is a system matrix that represents the operation of forward projection. As shown in the figure, forward projection is performed by converting each position x r For each line of response (LOR) in each direction φ, the sum of pixel values ​​of the pixels through which the line of response passes in the tomographic image 32 p(x r ,φ).

[0046]

number

[0047] Forward projection is analytically expressed by the following equation (2). Here, the coordinates (x, y) and the coordinates (x r ,y r ) is related by the following formula (3). φ is the azimuth angle that represents the direction of projection. x r ,x y are the x and y axes when the tomographic image is rotated by -φ. Forward projection is done by rotating the image by -φ degrees and then r By integrating the pixel values ​​along the axis, a one-dimensional projection p(x r ,φ) for 0≦φ<π to obtain a sinogram 31.

[0048]

number

[0049]

number

[0050] Back projection is the transpose of matrix A. T This is a process of projecting the sinogram 31 onto the real space using the above equation, which is expressed by the following equation (4). b is a backprojected image of the sinogram p. The backprojected image b is not equal to the tomographic image f.

[0051]

number

[0052] The projection process performed by the projection combination unit 24 (projection process from sinogram space to real space for any of the feature maps in the encoder unit 21) corresponds to the backprojection described above. By providing such a projection combination unit 24, it is expected that the encoder unit 21 will learn processing on sinograms, and the decoder unit 22 will learn processing on tomographic images. It is expected that learning of an algorithm for creating tomographic images from sinograms will be promoted without remaining in memory of pairs of sinograms and tomographic images used in learning.

[0053] Fig. 4 is a diagram showing another example of the configuration of a CNN. The CNN shown in this figure differs from the CNN shown in Fig. 2 in that it further includes a projection unit 25 and an adder unit 26. The projection unit 25 performs a projection process (back projection) from a sinogram space to a real space on a sinogram (measurement image) 31 input to the encoder unit 21, and creates a projection image 33, which is an image after the projection. The adder unit 26 adds the projection image 33 to the output image from the decoder unit 22, and creates a tomographic image 32.

[0054] In such a configuration, when the CNN training unit 15 trains the CNN using the sinograms 31 and tomographic images 32 of each of a plurality of subjects, the CNN training unit 15 trains the CNN on the difference between the projection image 33 obtained by backprojecting the sinogram 31 and the tomographic image 32 (residual training). The pixel-by-pixel difference between the projection image 33 and the tomographic image 32 can be not only a positive value but also a negative value, and has a distribution that can approximate a normal distribution centered on the value 0. This makes it easy to perform CNN training using the least squares method.

[0055] Next, the results of a simulation performed by creating simulation data by a numerical simulation using a digital brain phantom image will be described. The example uses the CNN shown in FIG. 2. The comparative example uses a technique for creating a tomographic image from a histogram using the CNN described in Non-Patent Document 1. In the comparative example, the CNN does not have anything equivalent to the projection and combination unit 24.

[0056] Twenty MRI brain segmentation images were downloaded from BrainWeb (https: / / brainweb.bic.mni.mcgill.ca / brainweb / ). The contrast ratio of gray matter, white matter, and cerebrospinal fluid (CSF) was set to 1:0.25:0.05. 18 F-FDG( 18 We created PET brain phantom images simulating F-fluorodeoxyglucose. The matrix size of each phantom image was 128 (X) × 128 (Y) × 70 (Z). Sinograms were created by forward projection of each phantom image. The matrix size of each sinogram was 128 (X r )×128(φ)×70(Z).

[0057] For each sample, two types were prepared: one without noise and one with noise. For the sinograms with noise, 20 pairs of sinograms and tomographic images (70 slices per sample) were created by adding Poisson noise equivalent to a total count of 10M per sample. Of these pairs, 18 samples were used as training data (total of 1260 slices) and 2 samples were used as test data (total of 140 slices).

[0058] In both the case of using noise-free samples and the case of using noise-containing samples, the CNN was trained using the training data (samples 1 to 18), and then the sinograms of the test data (samples 19 and 20) were input to the CNN to create a tomographic image, which was then compared with the original tomographic image. The tomographic images created by the CNN were evaluated using the peak signal-to-noise ratio (PSNR). The peak signal-to-noise ratio (PSNR) represents image quality in decibels (dB), and a higher value indicates better image quality.

[0059] 5 and 6 are diagrams showing tomographic images when a noise-free sample is used in the comparative example. FIG. 5(a) is a diagram showing an original tomographic image of sample 1. FIG. 5(b) is a diagram showing a tomographic image created by CNN based on the sinogram of sample 1. FIG. 6(a) is a diagram showing an original tomographic image of sample 19. FIG. 6(b) is a diagram showing a tomographic image created by CNN based on the sinogram of sample 19. The PSNR of the tomographic image (FIG. 5(b)) created by CNN based on the sinogram of the learning data (sample 1) was 35.30 dB. The PSNR of the tomographic image (FIG. 6(b)) created by CNN based on the sinogram of the test data (sample 19) was 22.63 dB.

[0060] 7 and 8 are diagrams showing tomographic images when a noise-free sample is used in the embodiment. FIG. 7(a) is a diagram showing an original tomographic image of sample 1. FIG. 7(b) is a diagram showing a tomographic image created by CNN based on the sinogram of sample 1. FIG. 8(a) is a diagram showing an original tomographic image of sample 19. FIG. 8(b) is a diagram showing a tomographic image created by CNN based on the sinogram of sample 19. The PSNR of the tomographic image (FIG. 7(b)) created by CNN based on the sinogram of the learning data (sample 1) was 35.08 dB. The PSNR of the tomographic image (FIG. 8(b)) created by CNN based on the sinogram of the test data (sample 19) was 35.05 dB.

[0061] 9 and 10 are diagrams showing tomographic images when a sample with noise is used in the comparative example. FIG. 9(a) is a diagram showing an original tomographic image of sample 1. FIG. 9(b) is a diagram showing a tomographic image created by CNN based on the sinogram of sample 1. FIG. 10(a) is a diagram showing an original tomographic image of sample 19. FIG. 10(b) is a diagram showing a tomographic image created by CNN based on the sinogram of sample 19. The PSNR of the tomographic image (FIG. 9(b)) created by CNN based on the sinogram of the learning data (sample 1) was 34.70 dB. The PSNR of the tomographic image (FIG. 10(b)) created by CNN based on the sinogram of the test data (sample 19) was 22.58 dB.

[0062] 11 and 12 are diagrams showing tomographic images when a sample with noise is used in the embodiment. FIG. 11(a) is a diagram showing an original tomographic image of sample 1. FIG. 11(b) is a diagram showing a tomographic image created by CNN based on the sinogram of sample 1. FIG. 12(a) is a diagram showing an original tomographic image of sample 19. FIG. 12(b) is a diagram showing a tomographic image created by CNN based on the sinogram of sample 19. The PSNR of the tomographic image (FIG. 11(b)) created by CNN based on the sinogram of the learning data (sample 1) was 30.34 dB. The PSNR of the tomographic image (FIG. 12(b)) created by CNN based on the sinogram of the test data (sample 19) was 28.26 dB.

[0063] Comparing the PSNRs of the tomographic images shown in Figs. 5 to 12, the following can be seen. When a tomographic image is created by CNN based on the sinogram of the learning data, regardless of the presence or absence of noise, the PSNR of the tomographic image created in the comparative example is as high as the PSNR of the tomographic image created in the embodiment. However, when a tomographic image is created by CNN based on the sinogram of the test data, regardless of the presence or absence of noise, the PSNR of the tomographic image created in the comparative example is low. This is considered to be due to the following reason. The CNN used in the comparative example only stores pairs of sinograms and tomographic images of the learning data, and when an unknown sinogram of test data is input, it outputs a tomographic image corresponding to the sinogram that is closest to the input sinogram among the sinograms used for learning. Therefore, it is considered that the comparative example cannot generalize to the unknown test data, and outputs a tomographic image of another person.

[0064] In contrast, in the embodiment, the PSNR of the tomographic image created by the CNN is high not only when the tomographic image is created by the CNN based on the sinogram of the learning data, but also when the tomographic image is created by the CNN based on the sinogram of the test data, regardless of the presence or absence of noise. In the embodiment, the CNN generalizes very well even to unknown test data. This is believed to be due to the following reason. In the embodiment, the CNN is provided with the projection combination unit 24, and therefore the CNN can learn an algorithm for creating a tomographic image from a sinogram without merely memorizing the pair of sinogram and tomographic image used in learning.

[0065] As described above, according to this embodiment, it is possible to suppress an increase in the amount of data required for CNN training and improve the performance of creating a tomographic image by CNN.

[0066] In the above-described embodiments and examples, the tomography apparatus is mainly a PET apparatus. However, the present invention can be applied to other tomography apparatuses in the same manner. When the tomography apparatus is a SPECT apparatus or an X-ray CT apparatus, the measurement space is a sinogram space, the measurement image is a sinogram, and the projection process from the measurement space to the real space is a back projection, as in the case of the PET apparatus. When the tomography apparatus is an MRI apparatus, the measurement space is a k-space, the measurement image is an image composed of k-space data, and the projection process from the measurement space to the real space is an inverse Fourier transform. [Explanation of symbols]

[0067] DESCRIPTION OF THE REFERENCE NUMERALS 1...tomography system, 2...tomography device, 10...image processing device, 11...memory unit, 12...measurement image creation unit, 13...correction unit, 14...CNN processing unit, 15...CNN learning unit, 21...encoder unit, 22...decoder unit, 23...bottleneck unit, 24...projection combination unit, 25...projection unit, 26...addition unit, 31...sinogram, 32...tomographic image, 33...projection image, 41-44...feature map, 51-54...feature map.

Claims

1. 1. An image processing device that creates a tomographic image of a subject in real space based on measurement data of the subject acquired by measurement using a tomography device, a measurement image creation unit that creates a measurement image in a measurement space different from a real space based on the measurement data; a CNN processing unit that inputs the measurement image into a convolutional neural network and creates the tomographic image by the convolutional neural network; Equipped with The convolutional neural network includes: an encoder unit that receives the measured image, updates a feature map by performing convolutional operations in each of a plurality of convolutional layers, and gradually reduces a spatial size of the feature map; a decoder unit provided at a subsequent stage of the encoder unit, the decoder unit updating a feature map by a convolution operation by each of a plurality of convolution layers and gradually expanding a spatial size of the feature map; a projection and combination unit that performs a projection process from a measurement space to a real space on any one of the feature maps in the encoder unit and combines the feature map after the projection process with a feature map of the same size in the decoder unit; Including, Image processing device.

2. Further comprising a CNN learning unit that trains the convolutional neural network using measurement images and tomographic images of each of a plurality of subjects. The image processing device according to claim 1 .

3. The CNN processing unit performs a projection process from a measurement space to a real space on the measurement image input to the encoder unit, and adds the image after the projection process to the output image from the decoder unit. The image processing device according to claim 1 .

4. The projection and combination unit performs a projection process from a measurement space to a real space on a feature map of a maximum size in the encoder unit, and combines the feature map after the projection process with a feature map of the same size in the decoder unit. The image processing device according to claim 1 .

5. The projection and combination unit performs a projection process from a measurement space to a real space on feature maps of all sizes in the encoder unit, and combines the feature maps after the projection process with feature maps of the same sizes in the decoder unit. The image processing device according to claim 1 .

6. a tomography apparatus for acquiring measurement data for producing a tomographic image of a subject; The image processing device according to any one of claims 1 to 5, which creates the tomographic image based on the measurement data; A tomography system comprising:

7. 1. An image processing method for creating a tomographic image of a subject in real space based on measurement data of the subject acquired by measurement using a tomography apparatus, comprising: a measurement image creating step of creating a measurement image in a measurement space different from a real space based on the measurement data; a CNN processing step of inputting the measurement image into a convolutional neural network and creating the tomographic image by the convolutional neural network; Equipped with The convolutional neural network includes: an encoder unit that receives the measured image, updates a feature map by performing convolutional operations in each of a plurality of convolutional layers, and gradually reduces a spatial size of the feature map; a decoder unit provided at a subsequent stage of the encoder unit, the decoder unit updating a feature map by a convolution operation by each of a plurality of convolution layers and gradually expanding a spatial size of the feature map; a projection and combination unit that performs a projection process from a measurement space to a real space on any one of the feature maps in the encoder unit and combines the feature map after the projection process with a feature map of the same size in the decoder unit; Including, Image processing methods.

8. Further comprising a CNN learning step of learning the convolutional neural network using measurement images and tomographic images of each of a plurality of subjects. The image processing method according to claim 7.

9. The CNN processing step performs a projection process from a measurement space to a real space on the measurement image input to the encoder unit, and adds the image after the projection process to an output image from the decoder unit. The image processing method according to claim 7.

10. The projection and combination unit performs a projection process from a measurement space to a real space on a feature map of a maximum size in the encoder unit, and combines the feature map after the projection process with a feature map of the same size in the decoder unit. The image processing method according to claim 7.

11. The projection and combination unit performs a projection process from a measurement space to a real space on feature maps of all sizes in the encoder unit, and combines the feature maps after the projection process with feature maps of the same sizes in the decoder unit. The image processing method according to claim 7.