Tomographic image estimation device and tomographic image estimation method

Abstract

To provide a tomographic image estimation device which can shorten the time of action restriction of a test subject necessary for acquiring a useful tomographic image when diagnosing the health condition of the test subject.SOLUTION: A tomographic image estimation device 1 receives the input of a tomographic image of a test subject in a first period after a medicine being a radioactive tracer is administered to the test subject as an input image I1, estimates the tomographic image of the test subject in a second period subsequent to the first period with a deep neural network (DNN), and outputs the estimated tomographic image as an output image I2. The tomographic image estimation device 1 can make the DNN learn by using a tomography image database 15. The tomographic image estimation device 1 comprises: an input unit 11; an estimation unit 12; an output unit 13; and a learning unit 14. The health condition of the test subject can be diagnosed on the basis of the estimated tomographic image of the test subject in the second period.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an apparatus and method for inferring tomographic images. [Background technology]

[0002] Radiation tomographic image acquisition devices such as PET (Positron Emission Tomography) devices and SPECT (Single Photon Emission Computed Tomography) devices can acquire tomographic images of a subject based on the distribution of gamma ray emission positions within the body of the subject to which a radioactive tracer drug has been administered. A doctor or technician can then diagnose the subject's health condition by viewing these tomographic images. Patent Document 1 discloses an invention that uses a neural network to diagnose the brain condition of a subject (e.g., a condition related to Alzheimer's disease) from a tomographic image of the brain of the subject.

[0003] After being administered to a subject, a radioactive tracer drug accumulates in a specific location in the subject's body over time. A radiation tomographic image acquisition device acquires a tomographic image of the subject based on the distribution of the drug accumulated in the specific location of the subject (i.e., the distribution of gamma ray emission positions). The tomographic image acquired in this way represents the condition of the specific location of the subject, and the subject's health condition can be diagnosed from this tomographic image.

[0004] Therefore, to diagnose the health condition of a subject, it is necessary to obtain tomographic images of the vicinity of a specific part of the subject for a certain period of time after the drug administration. When diagnosing Alzheimer's disease, it is necessary to obtain tomographic images of the subject's brain for a certain period of time (e.g., 20 minutes) after the drug administration. Based on the drug accumulation status in these tomographic images, it is possible to diagnose whether the subject has Alzheimer's disease. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2008 / 056638 Summary of the Invention [Problem to be solved by the invention]

[0006] It takes a long time after administration of a drug to obtain tomographic images useful for diagnosing the subject's health condition, which restricts the subject's movements for a long period of time, causing inconvenience to the subject, and also limits diagnostic throughput.

[0007] The present invention has been made to solve the above problems, and aims to provide a tomographic image inference device and a tomographic image inference method that can shorten the time a subject is restricted in their behavior required to obtain tomographic images that are useful for diagnosing the subject's health condition. [Means for solving the problem]

[0008] The tomographic image estimation device of the present invention comprises: (1) an input unit that inputs a tomographic image of a subject in a first period after a radioactive tracer drug is administered to the subject; (2) an estimation unit that estimates a tomographic image of the subject in a second period after the first period using a deep neural network based on the tomographic image input to the input unit; and (3) an output unit that outputs the results of estimation by the estimation unit.

[0009] It is preferable that the tomographic image estimation device of the present invention further includes a learning unit that trains the deep neural network using a database of tomographic images acquired for multiple subjects in a first period and a second period after drug administration.

[0010] In the tomographic image estimation device of the present invention, it is preferable that the estimation unit estimates the subject's health condition using a deep neural network based on tomographic images of the subject taken during both or either of the first and second periods after drug administration.

[0011] The tomographic image estimation method of the present invention comprises: (1) an input step of inputting a tomographic image of a subject in a first period after a radioactive tracer drug is administered to the subject; (2) an estimation step of estimating a tomographic image of the subject in a second period after the first period using a deep neural network based on the tomographic image input in the input step; and (3) an output step of outputting the result of the estimation in the estimation step.

[0012] It is preferable that the tomographic image estimation method of the present invention further includes a learning step of training a deep neural network using a database of tomographic images acquired for multiple subjects at a first period and a second period after drug administration.

[0013] In the tomographic image estimation method of the present invention, the estimation step preferably estimates the subject's health condition using a deep neural network based on tomographic images of the subject taken during both or either of the first and second periods after drug administration.

[0014] In the tomographic image estimation device or tomographic image estimation method of the present invention, the tomographic image of the subject is preferably a tomographic image of the brain, and the health condition of the subject is preferably related to Alzheimer's disease. [Effects of the Invention]

[0015] According to the present invention, it is possible to shorten the time required for restricting the behavior of a subject in order to obtain tomographic images useful for diagnosing the subject's health condition, and also to improve diagnostic throughput. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram showing the configuration of a tomographic image estimation device 1. As shown in FIG. [Figure 2] FIG. 2 is a table summarizing the details of the database of tomographic images used in the examples. [Figure 3]FIG. 3 is a diagram showing the structure of the DNN used in the examples. [Figure 4] 4 shows tomographic images of subject A (cognitively normal), where (a) shows a tomographic image from the first period, (b) shows a measured tomographic image, and (c) shows a predicted tomographic image. [Figure 5] 5 shows tomographic images of subject B (cognitively normal), where (a) shows a tomographic image from the first period, (b) shows a measured tomographic image, and (c) shows a predicted tomographic image. [Figure 6] 6 shows tomographic images of subject C (patient with frontotemporal dementia). (a) shows a tomographic image from the first period, (b) shows a measured tomographic image, and (c) shows a predicted tomographic image. [Figure 7] 7 shows tomographic images of subject D (patient with frontotemporal dementia). (a) shows a tomographic image from the first period, (b) shows a measured tomographic image, and (c) shows a predicted tomographic image. [Figure 8] 8 shows tomographic images of subject E (patient with mild cognitive impairment), where (a) shows a tomographic image from the first period, (b) shows a measured tomographic image, and (c) shows a predicted tomographic image. [Figure 9] 9 shows tomographic images of subject F (patient with mild cognitive impairment), where (a) shows a tomographic image from the first period, (b) shows a measured tomographic image, and (c) shows a predicted tomographic image. [Figure 10] 10 shows tomographic images of subject G (a patient with Alzheimer's disease), where (a) shows a tomographic image from the first period, (b) shows a measured tomographic image, and (c) shows a predicted tomographic image. [Figure 11] 11 shows tomographic images of subject H (patient with Alzheimer's disease), where (a) shows a tomographic image from the first period, (b) shows a measured tomographic image, and (c) shows a predicted tomographic image. [Figure 12] 12 shows tomographic images of subject I (cognitively normal), where (a) shows a tomographic image from the first period, (b) shows a measured tomographic image, and (c) shows a predicted tomographic image. [Figure 13]13 shows tomographic images of subject J (cognitively normal), where (a) shows a tomographic image from the first period, (b) shows a measured tomographic image, and (c) shows a predicted tomographic image. [Figure 14] 14 shows tomographic images of subject K (patient with frontotemporal dementia). (a) shows a tomographic image from the first period, (b) shows a measured tomographic image, and (c) shows a predicted tomographic image. [Figure 15] 15 shows tomographic images of subject L (patient with frontotemporal dementia), where (a) shows a tomographic image from the first period, (b) shows a measured tomographic image, and (c) shows a predicted tomographic image. [Figure 16] FIG. 16 is a table summarizing the results of assessment of amyloid β accumulation by two radiologists based on the measured tomographic images and estimated tomographic images of each of the subjects A to L. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are designated by the same reference numerals, and duplicate explanations will be omitted. The present invention is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.

[0018] FIG. 1 is a diagram showing the configuration of a tomographic image estimation device 1. The tomographic image estimation device 1 receives as input image I1 a tomographic image of a subject in a first period after the subject is administered a drug that is a radioactive tracer, estimates a tomographic image of the subject in a second period after the first period using a deep neural network (DNN), and outputs the estimated tomographic image as output image I2. The tomographic image estimation device 1 can also train the DNN using a tomographic image database 15. The tomographic image estimation device 1 includes an input unit 11, an estimation unit 12, an output unit 13, and a learning unit 14.

[0019] The input unit 11 inputs a tomographic image of the subject during a first period after drug administration as an input image I1. The tomographic image may be either a PET image acquired by a PET device or a SPECT image acquired by a SPECT device. The tomographic image may be a two-dimensional tomographic image of one slice or multiple slices, or may be a three-dimensional tomographic image.

[0020] The estimation unit 12 estimates a tomographic image of the subject in a second period subsequent to the first period using a DNN based on the tomographic image (input image I1) input to the input unit 11. Preferably, the estimation unit 12 estimates the subject's health condition using a DNN based on the tomographic images of the subject in both or either of the first and second periods after drug administration. The tomographic image of the subject may be a tomographic image of the brain. The subject's health condition may be, for example, the degree of dementia such as Alzheimer's disease, and can be expressed by a quantified Z-score, or by a risk level (requiring detailed examination, caution, follow-up, etc.).

[0021] This DNN is preferably a convolutional neural network (CNN). In a CNN, convolutional layers that extract features and pooling layers that compress the features are alternately arranged. The estimation unit 12 may perform DNN processing using a CPU (Central Processing Unit), but it is preferable to perform the processing using a DSP (Digital Signal Processor) or GPU (Graphics Processing Unit), which are capable of faster processing.

[0022] The output unit 13 outputs the result of the estimation by the estimation unit 12. The output unit 13 outputs the tomographic image of the subject for the second period estimated by the estimation unit 12 as an output image I2. Furthermore, if the estimation unit 12 also estimates the health condition of the subject, the output unit 13 also outputs the health condition of the subject estimated by the estimation unit 12. The output unit 13 preferably includes a display for displaying the image.

[0023] The learning unit 14 trains the DNN of the estimation unit 12 using a tomographic image database 15. The tomographic image database 15 stores data on tomographic images acquired for a plurality of subjects in a first period and a second period after drug administration. The learning unit 14 uses the tomographic image of each subject in the first period after drug administration as an input image I1, and trains the DNN of the estimation unit 12 based on an output image I2 corresponding to this input image I1 and the tomographic image of the subject in the second period after drug administration. This type of DNN learning is called deep learning.

[0024] A tomographic image estimation method using such a tomographic image estimation device 1 includes an input step by an input unit 11, an estimation step by an estimation unit 12, an output step by an output unit 13, and a learning step by a learning unit 14. Specifically, in the input step, a tomographic image of the subject in a first period after drug administration is input as an input image I1. In the estimation step, a tomographic image of the subject in a second period after the first period is estimated using a DNN based on the tomographic image (input image I1) input in the input step. Also, in the estimation step, it is preferable to estimate the subject's health condition using the DNN based on the tomographic images of the subject in both or either of the first and second periods after drug administration. In the output step, the tomographic image of the subject estimated in the estimation step is output as an output image I2, and the health condition of the subject estimated in the estimation step is also output. In the learning step, the DNN is trained using a tomographic image database 15.

[0025] Once the DNN has been trained in the training step, the series of steps of input, inference, and output can be repeated thereafter, so there is no need to perform the training step each time the series of steps of input, inference, and output is performed. For the same reason, if the DNN has already been trained, the training unit 14 is not necessary. However, even if the DNN has already been trained, if further training is to be performed to enable more accurate inference, the training step and training unit 14 may be present.

[0026] In the tomographic image estimation device or tomographic image estimation method of this embodiment, a DNN can be used to estimate a tomographic image of a subject in a second period after the first period based on a tomographic image of the subject in a first period after drug administration. Even if it is difficult to diagnose the subject's health condition based on the tomographic image in the first period, it becomes easy to diagnose the subject's health condition based on the tomographic image in the second period estimated by the DNN. Therefore, the time required for restricting the subject's behavior to obtain a tomographic image (a tomographic image in the second period) useful for diagnosing the subject's health condition can be shortened. Furthermore, diagnostic throughput can be improved.

[0027] Furthermore, if a doctor or other medical professional who interprets the tomographic images of the second period estimated by the DNN is unsure of how to diagnose the subject's health condition, the doctor or other medical professional may actually acquire tomographic images of the second period using a tomographic image acquisition device. In this case, the estimation of the tomographic images of the second period by the DNN and the interpretation by the doctor or other medical professional must be completed before the start of acquisition of the actual tomographic images of the second period.

[0028] Next, an example will be described. In the example described below, a tomographic image of the brain of a subject in a first period after drug administration was acquired, a tomographic image of the brain of the subject in a second period was estimated using a DNN based on the tomographic image of the brain of the subject in the first period, and whether or not the subject is suffering from Alzheimer's disease was diagnosed based on the estimated tomographic image of the brain of the subject in the second period.

[0029] The number of patients with dementia, including Alzheimer's disease, is increasing every year. It is estimated that by 2030, the number of patients with Alzheimer's disease worldwide will reach 76,000,000. It is believed that if Alzheimer's disease can be identified early, medical treatment can slow its progression, unlike other dementia-causing diseases such as Lewy body dementia and frontotemporal dementia. Early diagnosis of Alzheimer's disease also facilitates planning for long-term care and enrollment in clinical trials.

[0030] Alzheimer's disease is thought to develop due to the accumulation of amyloid beta in specific areas of the brain. 11 By administering C-Pittsburgh Compound-B (PiB) to a subject and obtaining a tomographic image of the subject's brain using a radiation tomography device, the accumulation of amyloid beta can be diagnosed based on the tomographic image, and Alzheimer's disease can also be diagnosed.

[0031] To obtain tomographic images useful for accurately diagnosing Alzheimer's disease using a radiation tomography system, it takes more than 50 minutes after administration of the drug (PiB) for the drug to bind to amyloid beta. During this time, the subject's movements are restricted, which places a heavy burden on patients suspected of having dementia and their caregivers, as well as on the doctors and other medical professionals who treat the patients.

[0032] The tomographic image estimation device or tomographic image estimation method of this embodiment can be applied to the diagnosis of Alzheimer's disease by acquiring tomographic images of the subject's brain during a first period (e.g., 0 to 20 minutes) after administration of a drug (PiB), and then estimating tomographic images of the subject's brain during a second period (e.g., 50 to 70 minutes) using a DNN based on the tomographic images from the first period. This allows for a diagnosis of whether the subject has Alzheimer's disease based on the estimated tomographic images of the subject's brain during the second period. This shortens the time the subject's behavior is restricted, reducing the burden on patients suspected of dementia, caregivers, doctors, and others. It also improves the efficiency of diagnosis.

[0033] Figure 2 is a table summarizing the breakdown of the tomographic image database used in the examples. The tomographic image database used in the examples consisted of dynamic brain PET images of 272 subjects (mean age 67.3 years, standard deviation of age 8.1 years, 159 women) collected by Hamamatsu Medical Center between October 2007 and February 2014. The 272 subjects consisted of 37 cognitively normal subjects, 44 patients with frontotemporal dementia, 23 patients with mild cognitive impairment, and 168 patients with Alzheimer's disease. Data from each patient group was divided into training data and test data. The training data consisted of 253 cases (mean age 67.2 years, standard deviation of age 8.0 years, 149 women), and the test data consisted of 19 cases (mean age 68.0 years, standard deviation of age 8.5 years, 10 women).

[0034] After intravenous injection of PiB (0.135 mCi / kg) into each subject, dynamic brain PET images were acquired using a Hamamatsu Photonics SHR12000 head PET scanner. The dynamic frame times for acquiring the dynamic brain PET images were 10 s x 6 frames, 60 s x 19 frames, and 300 s x 10 frames. A 120-min blank scan was performed to correct the sensitivity of each radiation detector in the PET scanner, and a 10-min transmission scan was performed to correct for attenuation for each subject. For each subject, the average of the dynamic brain PET images from 0 to 20 min after drug administration was used as the first period image, and the average of the dynamic brain PET images from 50 to 70 min after drug administration was used as the second period image.

[0035] A conditional generative adversarial network (C-GAN) was used for data augmentation. C-GAN uses a generator and a discriminator to improve accuracy. In this example, the generator generated tomographic images from a second period from tomographic images from a first period. The discriminator distinguished between tomographic images from the second period generated by the generator and tomographic images from the second period actually acquired. Competition between the generator and the discriminator enabled the creation of pairs of tomographic images from the first and second periods similar to pairs of tomographic images actually acquired from the first and second periods. As a result, not only pairs of tomographic images from the first and second periods actually acquired, but also pairs of tomographic images from the first and second periods created by the C-GAN were used as training data for the DNN. A training dataset of 253 subjects was used, and the DNN was trained using the Adam optimizer with a batch size of 8. The initial learning rate was set to 0.0002.

[0036] Figure 3 shows the structure of the DNN used in the examples. This DNN is an encoder-decoder CNN with a U-net structure. In the figure, tensors are represented by boxes, operations are represented by arrows, and the number of channels is written below each box.

[0037] The trained DNN was tested using a test dataset of 19 subjects. For each subject, the tomographic images from the first period were input to the trained DNN, and the trained DNN output the estimated tomographic images from the second period. Hereinafter, the tomographic images from the second period that were actually acquired will be referred to as "measured tomographic images," and the tomographic images from the second period estimated by the DNN will be referred to as "estimated tomographic images."

[0038] 4 to 15 are diagrams showing tomographic images, measured tomographic images, and estimated tomographic images for the first period of each of subjects A to L. In each diagram, (a) shows a tomographic image for the first period, (b) shows a measured tomographic image, and (c) shows an estimated tomographic image. FIG. 4 is a diagram showing tomographic images for subject A (cognitively normal). FIG. 5 is a diagram showing tomographic images for subject B (cognitively normal). FIG. 6 is a diagram showing tomographic images for subject C (frontotemporal dementia patient). FIG. 7 is a diagram showing tomographic images for subject D (frontotemporal dementia patient). FIG. 8 is a diagram showing tomographic images for subject E (mild cognitive impairment patient). FIG. 9 is a diagram showing tomographic images for subject F (mild cognitive impairment patient). FIG. 10 is a diagram showing tomographic images for subject G (Alzheimer's disease patient). FIG. 11 is a diagram showing tomographic images for subject H (Alzheimer's disease patient). FIG. 12 is a diagram showing tomographic images for subject I (cognitively normal). Fig. 13 shows cross-sectional images of subject J (cognitively normal), Fig. 14 shows cross-sectional images of subject K (frontotemporal dementia patient), and Fig. 15 shows cross-sectional images of subject L (frontotemporal dementia patient).

[0039] We evaluated the similarity between the measured and predicted tomographic images using statistical methods: root mean squared percentage error (RMSPE), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM). RMSPE is calculated by dividing the difference between the correct and predicted values ​​by the correct value for all data, and then taking the square root of this average. The RMSPE was 6.3% (±1.9%). PSNR is the ratio of the maximum possible power of a signal to the power of degrading noise that may affect the fidelity of its representation and is expressed in logarithmic form to approximate human visual perception. The PSNR was 21.8 dB (±2.2 dB). SSIM is a measure of structural similarity between images that incorporates not only single pixel values ​​but also calculates the mean, variance, and covariance with surrounding pixels. The SSIM was 0.45 (±0.04). Both indices indicated a high degree of similarity between the measured and estimated tomographic images.

[0040] 16 is a table summarizing the results of the assessment of amyloid-β accumulation by two radiologists based on the measured and estimated tomographic images of each of subjects A to L. The two radiologists interpreted the measured and estimated tomographic images for each subject and determined whether the amyloid-β accumulation was positive or negative.

[0041] The agreement between the judgment results based on the measured tomographic images and the judgment results based on the estimated tomographic images was evaluated. The agreement was evaluated using Cohen's kappa coefficient. The agreement between the judgment results by the two radiologists was 79% (kappa = 0.60) and 79% (kappa = 0.59), respectively. The agreement between the judgment results based on the measured tomographic images was 89% (kappa = 0.79) between the two radiologists. The measured tomographic images and the estimated tomographic images were evaluated to be very similar.

[0042] Alzheimer's disease patients, who had amyloid-β accumulation in the frontal lobe and periphery, could be easily distinguished from cognitively normal individuals and patients with frontotemporal dementia, who had no amyloid-β accumulation in either the actual or estimated tomographic images.

[0043] The results of the assessment based on the measured tomographic images differed from those based on the estimated tomographic images for Subject I (cognitively normal, Figure 12), Subject J (cognitively normal, Figure 13), Subject K (frontotemporal dementia patient, Figure 14), and Subject L (frontotemporal dementia patient, Figure 15). For these Subjects I to L, no accumulation of PiB was observed in the measured tomographic images, and the assessment results based on the measured tomographic images were negative, but no accumulation of PiB was observed in the frontal lobe or temporal lobe in the estimated tomographic images, and the assessment results based on the estimated tomographic images were positive.

[0044] We also evaluated whether two radiologists could distinguish between measured and estimated tomographic images. The two radiologists judged whether the tomographic images they read were measured or estimated tomographic images, and calculated the accuracy rate for this judgment. The accuracy rates for the two radiologists were 55% and 47%, respectively. This showed that radiologists were almost unable to distinguish between measured and estimated tomographic images.

[0045] As described above, according to this embodiment, based on the tomographic images of the subject taken in the first period after drug administration, the DNN can estimate the tomographic images of the subject taken in the second period after the first period (estimated tomographic images). Even if it is difficult to diagnose the subject's health condition based on the tomographic images taken in the first period, it becomes easy to diagnose the subject's health condition based on the estimated tomographic images. Therefore, the time required for restricting the subject's behavior to obtain tomographic images (tomographic images taken in the second period) that are useful for diagnosing the subject's health condition can be shortened. Furthermore, the throughput of diagnosis can be improved. [Explanation of symbols]

[0046] 1... tomographic image estimation device, 11... input unit, 12... estimation unit, 13... output unit, 14... learning unit, 15... tomographic image database.

Claims

1. an input unit for inputting a tomographic image of the subject during a first period after a drug that is a radioactive tracer is administered to the subject; an estimation unit that estimates a tomographic image of the subject in a second period after the first period using a deep neural network based on the tomographic image input to the input unit; an output unit that outputs a result of the estimation by the estimation unit; Equipped with The tomographic image of the subject in the second period estimated by the estimation unit is a tomographic image based on a drug administration before the first period in the second period in which no new drug administration has been performed since the drug administration before the first period. Tomographic image estimation device.

2. The tomographic image inference device according to claim 1, further comprising a learning unit that trains the deep neural network using a database of tomographic images acquired for a plurality of subjects during the first period and the second period after drug administration.

3. The estimation unit estimates the health state of the subject using a deep neural network based on tomographic images of the subject taken during both or either of the first period and the second period after drug administration.

3. The tomographic image estimation device according to claim 1 or 2.

4. The tomographic image of the subject is a tomographic image of the brain. The tomographic image estimation device according to any one of claims 1 to 3.

5. the tomographic image of the subject is a tomographic image of the brain, the subject's health condition is related to Alzheimer's disease; The tomographic image estimation device according to claim 3.

6. an input step of inputting a tomographic image of the subject during a first period after a drug that is a radioactive tracer is administered to the subject; an estimation step of estimating, by a deep neural network, a tomographic image of the subject in a second period that is subsequent to the first period, based on the tomographic image input in the input step; an output step of outputting a result of the inference in the inference step; Equipped with The tomographic image of the subject in the second period estimated in the estimation step is a tomographic image based on a drug administration before the first period in the second period in which no new drug administration has been performed since the drug administration before the first period. Tomographic image estimation method.

7. The tomographic image estimation method according to claim 6, further comprising a learning step of training the deep neural network using a database of tomographic images acquired for a plurality of subjects during the first period and the second period after drug administration.

8. the estimation step estimates a Z-score, which numerically represents the health state of the subject, using a deep neural network based on tomographic images of the subject taken during both or either of the first period and the second period after drug administration; The tomographic image estimation method according to claim 6 or 7.

9. The tomographic image of the subject is a tomographic image of the brain. The tomographic image estimation method according to any one of claims 6 to 8.

10. the tomographic image of the subject is a tomographic image of the brain, The Z-score is for Alzheimer's disease. The tomographic image estimation method according to claim 8.

Images