Information processing device, information processing method, program, learned model, and learning model generation method

Abstract

To provide an information processing device, an information processing method, a program, a learned model, and a learning model generation method capable of accurately calculating clinical parameters from an image that can be acquired by a simple or low-load method.SOLUTION: An information processing device includes one or more processors and one or more storage devices for storing a program. The program includes an image generation model that has learned to generate a second image simulating an image acquired by an imaging protocol different from that of a first image from the input first image. The image generation model is a model that has learned so that clinical parameters calculated from a generative image output by the image generation model get close to correct-answer clinical parameters by machine learning using a plurality of pieces of training data in which a training image captured by a first imaging protocol and a correct-answer clinical parameter calculated from a corresponding image captured by a second imaging protocol for the same subject using the same kind of modality are associated with each other.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, a program, a trained model, and a trained model generation method, and in particular to an image processing technique and a machine learning technique for calculating clinical parameters from medical images. [Background technology]

[0002] A common method for detecting cardiovascular disease (CVD) is to take computed tomography (CT) images of the chest region and quantify calcification in the coronary arteries and heart. Coronary artery calcification (CAC) scoring is the most characteristic and standard marker for CVD detection. The severity of CVD is evaluated based on the CAC score, and treatment is prescribed. Currently, when measuring the CAC score, ECG-gated CT images are obtained by a special imaging method synchronized with the electrocardiogram (ECG) to eliminate cardiac motion artifacts.

[0003] In contrast, Non-Patent Document 1 discloses a technique for classifying CAC severity into four stages with limited accuracy from non-ECG gated CT images obtained by an imaging method not synchronized with an ECG. The method described in Non-Patent Document 1 first uses a Convolutional Neural Network (CNN) to segment calcium regions from non-ECG gated CT images. Next, features such as the mean CT value and standard deviation are obtained from the segmented calcium regions, and a model is trained using another machine learning method to output a CAC severity stage that is the same as the severity stage identified in the corresponding ECG gated CT image.

[0004] In addition, regarding a technology for supporting image diagnosis, Patent Document 1 proposes a system for estimating images such as PET images using drugs from images such as MRI (Magnetic Resonance Imaging) images that can be taken without using radioactive drugs, in consideration of the disadvantages of images such as PET (Positron Emission Tomography) images taken using radioactive drugs, and using the images for diagnosis. The system described in Patent Document 1 has an acquisition unit for acquiring examinee information including real image data of an MR image including at least a reference region including a part of an evaluation target region of the examinee, and an information providing unit for providing diagnostic support information based on pseudo PET image data of an evaluation target region generated from real image data of an individual MR image of the examinee by an image processing model machine-learned to generate pseudo PET image data of an evaluation target region from real image data of an MR image of a reference region based on training data including real image data of an MR image of a plurality of subjects and real image data of a PET image including an evaluation target region. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6746160 [Non-patent literature]

[0006] [Non-Patent Document 1] Eng, D., Chute, C., Khandwala, N. et al. “Automated coronary calcium scoring using deep learning with multicenter external validation.” npj Digit. Med. 4, 88 (2021). Summary of the Invention [Problem to be solved by the invention]

[0007] Although ECG-gated imaging can obtain finer images than non-gated ECG imaging, the imaging method is complicated and requires a special imaging system. In addition, ECG-gated imaging increases the radiation dose to the subject compared to non-gated ECG imaging. From the viewpoint of reducing the radiation dose and the burden on the subject, it is desirable to measure the CAC score using regular chest CT images used for screening of lung cancer and respiratory diseases.

[0008] However, normal non-ECG-gated CT images obtained by taking images unsynchronized with the ECG are unclear (blurred) images due to cardiac motion, making it difficult to accurately measure the CAC score from non-ECG-gated CT images. Therefore, there is a need to develop a technology that can perform CAC scoring with sufficient accuracy from non-ECG-gated CT images.

[0009] In this regard, although Non-Patent Document 1 can estimate the CAC severity stage using non-ECG-gated CT images, it cannot calculate the CAC score with the same accuracy as ECG-gated CT images. In addition, the method for calculating the CAC severity described in Non-Patent Document 1 is a black box method using a so-called "unexplainable artificial intelligence (AI)" that cannot explain the process (why) by which the result was derived.

[0010] The above-mentioned issues are not limited to CT images and CAC scores, but are common when calculating various clinical parameters using images from other modalities. In response to these issues, technology is needed that can achieve the same diagnostic accuracy as images that can only be obtained using complex imaging methods, imaging methods that require expensive high-performance imaging systems, or imaging methods that are burdensome to the subject, using images that can be obtained more simply, at lower cost, or with less burden.

[0011] The present disclosure has been made in consideration of the above circumstances, and aims to provide an information processing device, an information processing method, a program, a trained model, and a trained model generation method that enable accurate calculation of clinical parameters from images that can be obtained relatively easily or with low load. [Means for solving the problem]

[0012] An information processing device according to a first aspect of the present disclosure includes one or more processors and one or more storage devices that store programs to be executed by the one or more processors, the program including an image generation model trained to generate a second image from an input first image that imitates an image obtained by an imaging protocol different from the first image, the image generation model being a model trained to approximate clinical parameters calculated from a generated image output by the image generation model in response to an input of a training image by machine learning using a plurality of training data in which training images captured by the first imaging protocol are associated with correct clinical parameters calculated from corresponding images captured by a second imaging protocol different from the first imaging protocol using the same type of modality as that used to capture the training images. The one or more processors accept input of a first image captured by the first imaging protocol, generate a second image from the first image by the image generation model, and calculate clinical parameters from the second image.

[0013] The term "imaging protocol" encompasses concepts of terms such as imaging method, imaging conditions, and scan settings.

[0014] According to the present disclosure, clinical parameters can be calculated from a first image captured by a first imaging protocol with the same accuracy as an image captured by a second imaging protocol of the same modality. The first imaging protocol may be a simpler imaging method than the second imaging protocol, and may be a less expensive system than a system required for imaging by the second imaging protocol. In addition, the first imaging protocol may be less burdensome for a subject than the second imaging protocol.

[0015] An information processing device according to a second aspect may be configured in the information processing device according to the first aspect, wherein the one or more processors divide the first image into anatomical regions and calculate clinical parameters for each of the divided anatomical regions.

[0016] An information processing device according to a third aspect is an information processing device according to the second aspect, in which the training image and the corresponding image are each divided into anatomical regions, and correct clinical parameters are calculated for each divided anatomical region of the corresponding image, and the image generation model may be a model trained so that each clinical parameter calculated from the generated image for each divided anatomical region of the training image approaches the correct clinical parameters for each anatomical region of the corresponding image.

[0017] An information processing device according to a fourth aspect may be the information processing device of the second or third aspect, wherein the divided anatomical region is a perfusion region of a coronary artery.

[0018] An information processing device according to a fifth aspect is the information processing device according to any one of the second to fourth aspects, wherein the clinical parameter is a calcium volume for each main branch of a coronary artery.

[0019] An information processing device according to a sixth aspect is an information processing device according to any one of the first to fifth aspects, wherein the first image is a non-ECG-synchronized CT image obtained by ECG-synchronized imaging, and the second image is an image simulating an ECG-synchronized CT image obtained by ECG-synchronized imaging.

[0020] An information processing device according to a seventh aspect is the information processing device according to any one of the first to sixth aspects, wherein the training images may be taken under lower radiation conditions than the corresponding images.

[0021] An information processing device according to an eighth aspect is the information processing device according to any one of the first to sixth aspects, wherein the first imaging protocol has a lower radiation dose than the second imaging protocol.

[0022] An information processing device according to a ninth aspect is the information processing device according to any one of the first to eighth aspects, wherein the first imaging protocol has a slower scan speed than the second imaging protocol.

[0023] An information processing device according to a tenth aspect may be the information processing device according to any one of the first to ninth aspects, wherein the clinical parameter is a calcification score.

[0024] An information processing device according to an eleventh aspect may be the information processing device according to any one of the first to ninth aspects, in which the clinical parameter is a calcium volume in cardiovascular blood vessels.

[0025] An information processing device according to a twelfth aspect may be the information processing device according to any one of the first to ninth aspects, wherein the clinical parameter is a severity of coronary artery calcification.

[0026] An information processing device according to a thirteenth aspect is the information processing device according to any one of the first to twelfth aspects, in which the image generation model may be configured using a neural network.

[0027] An information processing method according to a fourteenth aspect is an information processing method executed by one or more processors, the one or more processors including: acquiring a first image taken by a first shooting protocol; inputting the first image into an image generation model; generating a second image from the first image by the image generation model, the second image simulating an image obtained when the same subject as the first image is photographed by a second shooting protocol different from the first shooting protocol using the same modality as the modality used to photograph the first image; and calculating clinical parameters from the second image, the image generation model being a model trained using a plurality of training data in which a training image taken by the first shooting protocol is associated with a correct clinical parameter calculated from a corresponding image taken by the second shooting protocol using the same modality as the modality used to photograph the training image, so that the clinical parameter calculated from the generated image output by the image generation model in response to the input of the training image approaches the correct clinical parameter.

[0028] The information processing method according to the fourteenth aspect may have a configuration including the same specific aspect as the information processing device according to any one of the second to thirteenth aspects.

[0029] A program according to a fifteenth aspect of the present invention is a program that causes a computer to realize the following functions: acquiring a first image taken by a first shooting protocol; inputting the first image into an image generation model and generating a second image from the first image using the image generation model, the second image simulating an image obtained when the same subject as the first image is photographed by a second shooting protocol different from the first shooting protocol using the same modality as the modality used to photograph the first image; and calculating clinical parameters from the second image. The image generation model is a model that has been trained by machine learning using a plurality of training data in which training images taken by the first shooting protocol are associated with correct clinical parameters calculated from corresponding images taken by the second shooting protocol using the same modality as the modality used to photograph the training image, so that clinical parameters calculated from generated images output by the image generation model in response to input of a training image approach the correct clinical parameters.

[0030] The program according to the fifteenth aspect may have a configuration including the same specific aspect as the information processing device according to any one of the second to thirteenth aspects.

[0031] The trained model of the 16th aspect is a trained model that enables a computer to perform a function of generating a pseudo image from an input image that imitates an image obtained by an imaging protocol different from the input image, and is a trained model that has been trained by machine learning using multiple training data in which training images captured by a first imaging protocol are associated with correct clinical parameters calculated from corresponding images captured by the same subject as the training images using the same type of modality as that used to capture the training images and a second imaging protocol different from the first imaging protocol, so that the clinical parameters calculated from the generated images output by the trained model in response to input training images approach the correct clinical parameters.

[0032] The trained model according to the sixteenth aspect may be configured to include the same specific aspect as the information processing device according to any one of the second to thirteenth aspects.

[0033] A learning model generation method according to a 17th aspect is a learning model generation method for generating a learning model that enables a computer to perform a function of generating a pseudo-image that imitates an image obtained by a shooting protocol different from an input image, and includes a system including one or more processors performing machine learning using a plurality of training data in which training images captured by a first shooting protocol are associated with correct clinical parameters calculated from corresponding images captured by a second shooting protocol different from the first shooting protocol using the same type of modality as the modality used to capture the training images, inputting the training images into the learning model, calculating clinical parameters from the generated images output from the learning model, and learning the learning model so that the clinical parameters calculated from the generated images approach the correct clinical parameters.

[0034] The learning model generation method according to the seventeenth aspect may have a configuration including the same specific aspect as the information processing device according to any one of the second to thirteenth aspects. Effect of the Invention

[0035] According to the present disclosure, clinical parameters can be calculated based on a first image captured by a first imaging protocol with the same accuracy as an image captured by a second imaging protocol of the same modality. This makes it possible to achieve a diagnostic accuracy from an image captured by the first imaging protocol that is the same as a diagnostic accuracy based on an image captured by the second imaging protocol. The first imaging protocol can be a simpler imaging method than the second imaging protocol, and can be a less expensive system than a system required for imaging by the second imaging protocol. In addition, the first imaging protocol can be a lower burden on the subject than the second imaging protocol. [Brief description of the drawings]

[0036] [Figure 1] FIG. 1 is an explanatory diagram illustrating a schematic diagram of the heart and coronary arteries. [Diagram 2] FIG. 2 is an example of a CT image of the chest region including the heart. [Diagram 3] FIG. 3 is an explanatory diagram showing an example of a CAC score measured from an ECG-gated CT image. [Figure 4] FIG. 4 is an explanatory diagram showing an example of a CAC score measured from a non-ECG-gated CT image of the same subject as the ECG-gated CT image shown in FIG. [Diagram 5] FIG. 5 is a block diagram showing a functional configuration of the information processing device according to the first embodiment. [Figure 6] FIG. 6 is a block diagram illustrating an example of a hardware configuration of the information processing device according to the first embodiment. [Figure 7] FIG. 7 is an explanatory diagram showing an overview of a machine learning method for generating an image generation model applied to the first embodiment. [Figure 8] FIG. 8 is a block diagram illustrating a functional configuration of a machine learning device applied to the first embodiment. [Figure 9] FIG. 9 is a block diagram illustrating an example of a hardware configuration of a machine learning device applied to the first embodiment. [Figure 10] FIG. 10 is a block diagram showing the functional configuration of a training data generation device that performs processing to generate training data used in the machine learning method of the first embodiment. [Figure 11] FIG. 11 is a conceptual diagram of the training data set. [Figure 12] FIG. 12 is a flowchart showing an example of a machine learning method executed by the machine learning device. [Figure 13] FIG. 13 is a flowchart illustrating an example of an information processing method executed by the information processing device. [Figure 14] FIG. 14 is a block diagram showing a functional configuration of an information processing device according to the second embodiment. [Figure 15]FIG. 15 is a block diagram illustrating an example of a hardware configuration of an information processing device according to the second embodiment. [Figure 16] FIG. 16 is an explanatory diagram showing an overview of a machine learning method for generating an image generation model applied to the second embodiment. [Figure 17] FIG. 17 is a block diagram illustrating a functional configuration of a machine learning device applied to the second embodiment. [Figure 18] FIG. 18 is a block diagram illustrating an example of a hardware configuration of a machine learning device applied to the second embodiment. [Figure 19] FIG. 19 is a block diagram showing the functional configuration of a training data generation device that performs processing to generate training data used in the machine learning method of the second embodiment. [Figure 20] FIG. 20 is a flowchart illustrating an example of a machine learning method executed by the machine learning device applied to the second embodiment. [Figure 21] FIG. 21 is a flowchart showing an example of an information processing method executed by the information processing device according to the second embodiment. [Figure 22] FIG. 22 is a graph showing the relationship between the CAC score calculated from a non-ECG-gated CT image and the CAC score calculated from an ECG-gated CT image using the information processing device according to the second embodiment. [Diagram 23] FIG. 23 is a graph showing the relationship between the CAC score calculated from a non-ECG-gated CT image using an information processing device according to a comparative example and the CAC score calculated from an ECG-gated CT image. [Figure 24] FIG. 24 shows examples of an ECG-gated CT image, a non-ECG-gated CT image, and a pseudo image generated from the non-ECG-gated CT image, all of which are used to calculate the CAC score. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0037] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

[0038] 《CAC Scoring Explained》 Here, an example will be described in which the CAC score is measured from a CT image obtained by photographing the chest region including the heart of a subject using a CT device. The CT device is an example of a modality for photographing medical images. The CAC score is an example of a clinical parameter used for diagnosis. FIG. 1 is an explanatory diagram showing a schematic diagram of the heart and coronary arteries. The left diagram F1A in FIG. 1 is a diagram showing the heart from the front, and the right diagram F1B in FIG. 1 is a diagram showing the heart from above. The coronary arteries include four main branches: the left main coronary trunk (LM), the left anterior descending artery (LAD), the left circumflex artery (LCX), and the right coronary artery (RCA). In FIG. 1, "AA" stands for the ascending aorta, and "PT" stands for the pulmonary artery trunk.

[0039] CAC scoring measures the amount of calcium deposits in these four main branches of the coronary arteries. Calcium in the coronary arteries is a factor in causing heart disease. The risk of heart disease is correlated with the total CAC score, which is the sum of the calcium levels in each main branch of the coronary artery, and treatment methods are determined based on the calculated CAC score.

[0040] FIG. 2 is an example of a CT image of a chest region including the heart. Image F2A on the left side of FIG. 2 is an example of an ECG-gated CT image obtained by ECG-gated imaging. Image F2B on the right side of FIG. 2 is an example of a non-ECG-gated CT image obtained by ECG-gated imaging of the same subject. As is clear from comparing the two images, the ECG-gated CT image (left) is a fine image F2A in which the influence of cardiac motion is suppressed, whereas the non-ECG-gated CT image (right) is an unclear (blurred) image F2B due to cardiac motion during imaging. Note that both the non-ECG-gated CT image and the ECG-gated CT image are three-dimensional images composed of three-dimensional data obtained by successively capturing two-dimensional slice tomographic images. The term "image" includes the meaning of image data.

[0041] The ECG-gated imaging is an example of a "first imaging protocol" in the present disclosure. The ECG-unsynchronized imaging is an example of a "second imaging protocol" in the present disclosure. The ECG-gated CT image and the non-ECG-gated CT image are examples of images captured using the same type of modality.

[0042] Non-ECG-gated CT images can be taken under lower radiation dose conditions than ECG-gated CT images. In other words, ECG-gated imaging can be taken with lower radiation doses than ECG-gated imaging, and is an imaging method that places less strain on the subject. In addition, non-ECG-gated CT images can be taken at a slower scan speed than ECG-gated CT images.

[0043] 《Explanation of the assignment》 FIG. 3 is an explanatory diagram showing an example of a CAC score measured from an ECG-gated CT image F3A. In the ECG-gated CT image F3A, the heart region is divided into perfusion regions (dominant regions) of four main branches, and a CAC score is calculated for each main branch region. The perfusion region of each main branch may be automatically identified by image processing using, for example, a segmentation model trained by machine learning, or a doctor may manually specify the region while observing the image. Identification of whether or not a region is calcium can be determined by threshold processing of the CT value of each voxel. For example, a region with a CT value greater than 100 HU is determined to be a calcium region.

[0044] In the example shown in FIG. 3, the individual CAC scores of the coronary artery branches RCA, LM, LAD, and LCX are calculated as “0”, “0”, “90”, and “30”, respectively, and the total CAC score is calculated as “120”.

[0045] Fig. 4 is an explanatory diagram showing an example of CAC scores measured from a non-ECG-gated CT image F4A of the same subject as the ECG-gated CT image F3A shown in Fig. 3. When the CAC scores for each region of the coronary artery branch are calculated by performing threshold processing of the CT values ​​for the non-ECG-gated CT image F4A shown in Fig. 4 in the same manner as for the ECG-gated CT image F3A in Fig. 3, the individual CAC scores for the RCA, LM, LAD, and LCX coronary artery branches are calculated as "0", "0", "40", and "10", respectively, and the total CAC score is calculated as "50". The total CAC score is an example of the "calcification score" in the present disclosure.

[0046] As shown in FIG. 4, the CAC score calculated from the non-ECG-synchronized CT image F4A deviates from the CAC score calculated from the ECG-synchronized CT image F3A, and it is difficult to accurately (precisely) calculate a CAC score from the non-ECG-synchronized CT image F4A with the same degree of precision as the ECG-synchronized CT image F3A.

[0047] In Figures 3 and 4, an individual CAC score is calculated for each perfusion region of the four main branches, and the total CAC score is calculated from the sum of these scores, but the CAC score may be calculated by counting the number of voxels with a CT value of 100 HU or more from the entire heart without dividing the perfusion regions of the main branches. Note that "100 HU" is an example of a threshold value for determining calcium regions.

[0048] First Embodiment [Configuration of information processing device] 5 is a block diagram showing the functional configuration of the information processing device 10 according to the first embodiment. The information processing device 10 includes a data acquisition unit 12, an image generation model 14, a clinical parameter calculation unit 16, and a processing result output unit 18. Various functions of the information processing device 10 can be realized by a combination of computer hardware and software. The information processing device 10 may be configured by a computer system including one or more computers.

[0049] The data acquisition unit 12 acquires a non-ECG-gated CT image IMng as a processing target. The non-ECG-gated CT image IMng input via the data acquisition unit 12 is actual image data obtained by actually imaging a subject using a CT device.

[0050] The image generation model 14 is a trained model trained by machine learning to receive a non-ECG-gated CT image IMng and generate a pseudo-ECG-gated CT image IMsyn from the non-ECG-gated CT image IMng. The image generation model 14 is configured using a so-called encoder-decoder type fully convolutional neural network (FCN) structure.

[0051] The pseudo ECG-gated CT image IMsyn output from the image generation model 14 is a synthetic image artificially generated by image processing for use in calculating the CAC score. The pseudo ECG-gated CT image IMsyn is an intermediate image generated in the process of predicting the CAC score based on the input non-ECG-gated CT image IMng. The pseudo ECG-gated CT image IMsyn is generated as an artificial image simulating an ECG-gated CT image obtained by ECG-gated imaging, but it does not matter whether the pseudo image itself is similar to the characteristics of an ECG-gated CT image of an actual image obtained by actual ECG-gated imaging, as long as the target CAC score can be calculated with high accuracy from this pseudo image (intermediate image). The term "pseudo image" may be replaced with "corrected image" or "converted image".

[0052] The non-ECG-gated CT image IMng is an example of a "first image" in the present disclosure, and the pseudo-ECG-gated CT image IMsyn is an example of a "second image" in the present disclosure.

[0053] The image generation model 14 is a model trained using training data in which a non-ECG-gated CT image as a training image is associated (linked) with a correct CAC score calculated from an ECG-gated CT image of the same subject as the training image, so that a CAC score calculated from an output image (pseudo image) output by the model in response to an input of the training image approaches the correct CAC score. A machine learning method for obtaining the image generation model 14 will be described in detail later.

[0054] The clinical parameter calculation unit 16 calculates clinical parameters for diagnosis based on the pseudo ECG-gated CT image IMsyn generated by the image generation model 14. In this embodiment, the clinical parameter to be calculated is the CAC score.

[0055] The processing result output unit 18 outputs the processing result including the values ​​of the clinical parameters (calculation results of the clinical parameters) calculated by the clinical parameter calculation unit 16. The processing result output unit 18 may be configured to perform at least one of the following processes: displaying the processing result, recording the processing result in a database or the like, and printing the processing result.

[0056] 6 is a block diagram showing an example of a hardware configuration of the information processing device 10. The information processing device 10 includes a processor 102, a computer-readable medium 104 that is a non-transient tangible object, a communication interface 106, an input / output interface 108, and a bus 110. The processor 102 is connected to the computer-readable medium 104, the communication interface 106, and the input / output interface 108 via the bus 110. The form of the information processing device 10 is not particularly limited, and may be a server, a personal computer, a workstation, a tablet terminal, or the like. The information processing device 10 may also be a viewer terminal for image interpretation, or the like.

[0057] The processor 102 includes a central processing unit (CPU). The processor 102 may include a graphics processing unit (GPU). The computer-readable medium 104 includes a memory 112 that is a main storage device and a storage 114 that is an auxiliary storage device. The computer-readable medium 104 may be, for example, a semiconductor memory, a hard disk drive (HDD) device, a solid state drive (SSD) device, or a combination of a plurality of these. The computer-readable medium 104 is an example of a "storage device" in this disclosure.

[0058] The computer-readable medium 104 includes an input image storage area 120, a generated image storage area 122, and a processing result storage area 124. The computer-readable medium 104 also stores a plurality of programs and data, including an image generation model 14, a clinical parameter calculation program 160, a processing result output program 180, and a display control program 190. The term "program" includes the concept of a program module. The processor 102 functions as various processing units by executing instructions of the programs stored in the computer-readable medium 104.

[0059] The input image storage area 120 stores a non-ECG-gated CT image IMng input as a target image for processing. The generated image storage area 122 stores a pseudo ECG-gated CT image IMsyn, which is a generated image output from the image generation model 14. The clinical parameter calculation program 160 includes an instruction to execute a process of calculating clinical parameters from the pseudo ECG-gated CT image IMsyn. The processing result storage area 124 records information on the processing result including the values ​​of the clinical parameters calculated by the clinical parameter calculation program 160. The processing result output program 180 includes an instruction to execute a process of outputting the processing result including the values ​​of the clinical parameters. The display control program 190 includes an instruction to generate a display signal required for display output to the display device 154 and execute display control of the display device 154.

[0060] The information processing device 10 can be connected to an electric communication line (not shown) via the communication interface 106. The electric communication line may be a wide area communication line, a private network communication line, or a combination of these.

[0061] The information processing device 10 may include an input device 152 and a display device 154. The input device 152 is configured, for example, by a keyboard, a mouse, a multi-touch panel, or other pointing devices, or a voice input device, or an appropriate combination of these. The display device 154 is configured, for example, by a liquid crystal display, an organic electro-luminescence (OEL) display, or a projector, or an appropriate combination of these. The input device 152 and the display device 154 are connected to the processor 102 via the input / output interface 108.

[0062] [Overview of machine learning methods] FIG. 7 is an explanatory diagram showing an overview of a machine learning method for generating the image generation model 14 applied to the first embodiment.

[0063] The data set used for training includes a plurality of training data in which a non-ECG-gated CT image F7A obtained by imaging a subject and a CAC score GTS calculated from an ECG-gated CT image F7B obtained by imaging the same subject as the non-ECG-gated CT image F7A are associated with each other. The non-ECG-gated CT image F7A is a training image for input to the learning model 210. The CAC score GTS is a correct CAC score corresponding to the non-ECG-gated CT image F7A, and corresponds to teacher data (correct answer data) in supervised learning. The CAC score GTS may be, for example, a value obtained by counting the number of voxels in an area in the ECG-gated CT image F7B where the CT value is greater than 100 HU. In the case of the first embodiment, the CAC score GTS may be a total CAC score. The ECG-gated CT image F7B is an example of a "corresponding image" in the present disclosure.

[0064] The learning model 210 is composed of a full-layer convolutional neural network and is trained in the following manner.

[0065] [Step 1] A non-ECG-gated CT image F7A is input to the learning model 210 as a training image.

[0066] [Step 2] A non-ECG-gated CT image F7A is input and a generated image F7C is output from the learning model 210.

[0067] [Step 3] A CAC score PRS is calculated by threshold processing under the condition of a CT value > 100 HU for the generated image F7C output from the learning model 210. The CAC score PRS calculated from the generated image F7C is a CAC score predicted from the non-ECG-gated CT image F7A.

[0068] [Step 4] The machine learning device including a processor that executes machine learning processing calculates a loss from the CAC score PRS calculated from the generated image F7C and the correct CAC score GTS, and updates the model parameters of the learning model 210 so as to minimize the loss. The loss may be, for example, an L1 loss. Note that learning to minimize the loss means learning so that the CAC score PRS calculated from the generated image F7C approaches the correct CAC score GTS (so that the error becomes smaller).

[0069] By repeating the above steps 1 to 4 using multiple training data, the model parameters of the learning model 210 are optimized, and the learning model 210 is trained to generate a generated image F7C for which a CAC score PRS is calculated with the same accuracy as the correct CAC score.

[0070] 8 is a block diagram showing the functional configuration of the machine learning device 30. The machine learning device 30 includes a learning model 210, a clinical parameter calculation unit 220, a loss calculation unit 230, a model parameter update amount calculation unit 232, and a model parameter update unit 234. The functions of each processing unit of the machine learning device 30 can be realized by a combination of computer hardware and software. The functions of each unit of the machine learning device 30 may be realized by one computer, or may be realized by sharing the processing functions among two or more computers.

[0071] The training data includes training images TIj, which are non-ECG-gated CT images, and corresponding correct clinical parameters GTj. The subscript j represents an index number for identifying a plurality of training data. In the first embodiment, the correct clinical parameters GTj may be CAC scores calculated from ECG-gated CT images of the same subject as the training images TIj.

[0072] The learning model 210 receives training images TIj, which are non-ECG-synchronized CT images, and outputs generated images SIj. The clinical parameter calculation unit 220 calculates clinical parameters PPj from the generated images SIj. The clinical parameters PPj may be total CAC scores calculated from the generated images SIj by threshold processing. The clinical parameters PPj calculated from the generated images SIj correspond to predicted values ​​of clinical parameters predicted (estimated) from the training images TIj.

[0073] The loss calculation section 230 calculates a loss indicating an error between the clinical parameter PPj calculated by the clinical parameter calculation section 220 and the correct clinical parameter GTj.

[0074] Based on the calculated loss, the model parameter update amount calculation unit 232 calculates the update amount of the model parameters in the learning model 210. The model parameters include the filter coefficients (the weights of connections between nodes) of the filters used in the processing of each layer of the CNN, the biases of the nodes, and the like.

[0075] The model parameter update unit 234 updates the model parameters of the learning model 210 according to the update amount calculated by the model parameter update amount calculation unit 232. The model parameter update amount calculation unit 232 and the model parameter update unit 234 optimize the model parameters by a method such as Stochastic Gradient Descent (SGD).

[0076] FIG. 9 is a block diagram showing an example of a hardware configuration of the machine learning device 30. The machine learning device 30 includes a processor 302, a computer-readable medium 304 that is a non-transient tangible object, a communication interface 306, an input / output interface 308, and a bus 310. The computer-readable medium 304 includes a memory 312 and a storage 314. The processor 302 is connected to the computer-readable medium 304, the communication interface 306, and the input / output interface 308 via the bus 310. The input device 352 and the display device 354 are connected to the bus 310 via the input / output interface 308. The hardware configuration of the machine learning device 30 may be the same as the corresponding elements of the information processing device 10 described in FIG. 6. The machine learning device 30 may be in the form of a server, a personal computer, or a workstation.

[0077] The machine learning device 30 is connected to an electric communication line (not shown) via the communication interface 306, and is communicatively connected to external devices such as a training data storage unit 550. The training data storage unit 550 includes a storage in which a training data set including a plurality of training data is stored. Note that the training data storage unit 550 may be built in the storage 314 in the machine learning device 30.

[0078] The computer-readable medium 304 stores a plurality of programs, data, and the like, including a learning processing program 330 and a display control program 340. The learning processing program 330 includes a data acquisition program 400, a learning model 210, a clinical parameter calculation program 420, a loss calculation program 430, and an optimizer 440. The data acquisition program 400 includes an instruction to execute a process of acquiring training data from a training data storage unit 550. The clinical parameter calculation program 420 may be similar to the clinical parameter calculation program 160 described in FIG. 6.

[0079] The loss calculation program 430 includes an instruction to execute a process of calculating a loss indicating an error between the clinical parameter calculated by the clinical parameter calculation program 420 and the correct clinical parameter corresponding to the training image. The optimizer 440 includes an instruction to execute a process of calculating an update amount of the model parameter from the calculated loss and updating the model parameter of the learning model 210.

[0080] [How to generate training data] FIG. 10 is a block diagram showing the functional configuration of a training data generation device 50 that performs processing to generate training data used in the machine learning method of the first embodiment.

[0081] The training data generation device 50 includes a clinical parameter calculation unit 520 and an association processing unit 530. In order to generate training data, first, a plurality of image pairs of a non-ECG-gated CT image NIj and an ECG-gated CT image GIj including a region of the same part of the same subject are prepared.

[0082] The non-ECG-gated CT image NIj becomes a training image TIj for input to the learning model 210. The clinical parameter calculation unit 520 calculates clinical parameters from the ECG-gated CT image GIj. The association processing unit 530 associates (links) the clinical parameters CPj calculated from the ECG-gated CT image GIj with the non-ECG-gated CT image NIj. That is, the association processing unit 530 associates the clinical parameters CPj calculated from the ECG-gated CT image GIj with the non-ECG-gated CT image NIj as the correct clinical parameters GTj corresponding to the non-ECG-gated CT image NIj (training image TIj).

[0083] The non-ECG-gated CT images NIj and clinical parameters CPj associated by the association processing unit 530 are stored in the training data storage unit 550 as training images TIj and correct clinical parameters GTj. The training data storage unit 550 may be included in the training data generation device 50, or may be a device separate from the training data generation device 50. By performing similar processing on image pairs captured for each of a plurality of subjects, a plurality of sets of training data are obtained. The function of the training data generation device 50 may be incorporated in the machine learning device 30.

[0084] Fig. 11 is a conceptual diagram of a training data set stored in the training data storage unit 550. As shown in Fig. 11, the training data set includes a plurality of training data in which input training images TIj are associated with values ​​of CAC scores, which are corresponding correct clinical parameters GTj. It is preferable to prepare a training data set as shown in Fig. 11 in advance and obtain samples from the training data set to train the learning model 210. It is also possible to generate training data by calculating correct clinical parameters from ECG-gated CT images GIj during training of the learning model 210.

[0085] [Flowchart of the machine learning method executed by the machine learning device 30] 12 is a flowchart showing an example of a machine learning method executed by the machine learning device 30. In step S102, the processor 302 acquires training data.

[0086] In step S104, the processor 302 inputs the training images TIj of the training data to the learning model 210, and causes the learning model 210 to generate a pseudo image.

[0087] In step S106, the processor 302 calculates clinical parameters from the pseudo image output by the learning model 210.

[0088] In step S108, the processor 302 calculates a loss indicating the error between the clinical parameters calculated from the pseudo image and the ground truth clinical parameters GTj associated with the training images TIj.

[0089] In step S110, the processor 302 calculates the update amount of the model parameters of the learning model 210 based on the calculated loss, and updates the model parameters. Note that the operations from step S102 to step S110 may be performed in mini-batch units.

[0090] In step S112, the processor 302 determines whether or not to end the learning. The learning termination condition may be determined based on the loss value or the number of updates of the model parameters. As a method based on the loss value, for example, the learning termination condition may be that the loss has converged within a specified range. As a method based on the number of updates, for example, the learning termination condition may be that the number of updates has reached a specified number. Alternatively, a data set for evaluating the performance of the model may be prepared separately from the training data, and whether or not to terminate the learning may be determined based on an evaluation value using the evaluation data.

[0091] If the determination result in step S112 is a No determination, the processor 302 returns to step S102 and continues the learning process. On the other hand, if the determination result in step S112 is a Yes determination, the processor 302 ends the flowchart in FIG.

[0092] In this way, the trained learning model 210 is incorporated into the information processing device 10 as the image generation model 14. The machine learning method executed by the machine learning device 30 can be understood as a method for generating the image generation model 14, and is an example of a learning model generation method in the present disclosure.

[0093] [Flowchart of information processing method executed by information processing device 10] 13 is a flowchart showing an example of an information processing method executed by the information processing device 10. In step S202, the processor 102 acquires a target image to be processed. In this case, the target image is a non-ECG-gated CT image. The processor 102 may automatically acquire the target image from an image storage server (not shown) or the like, or may accept an input of the target image via a user interface and acquire the specified image. The image storage server may be, for example, a DICOM (Digital Imaging and Communications in Medicine) server.

[0094] In step S204, the processor 102 inputs the target image to the image generation model 14, and the image generation model 14 generates a pseudo image.

[0095] In step S206, the processor 102 calculates clinical parameters from the pseudo image.

[0096] In step S208, the processor 102 outputs the processing result including the calculated value of the clinical parameter.

[0097] After step S208, the processor 102 ends the flowchart in FIG.

[0098] Second Embodiment In the second embodiment, an example will be described in which a CAC score is calculated for each perfusion region of four main branches of a coronary artery.

[0099] [Configuration of information processing device] Fig. 14 is a block diagram showing the functional configuration of an information processing device 100 according to the second embodiment. In Fig. 14, elements that are the same as or similar to those in the configuration shown in Fig. 5 are given the same reference numerals, and duplicated explanations will be omitted. Differences between the configuration shown in Fig. 14 and Fig. 5 will be explained.

[0100] The information processing device 100 shown in Fig. 14 includes an anatomical region dividing unit 15 that divides a non-ECG-gated CT image input via the data acquiring unit 12 into perfusion regions of four main branches of a coronary artery. The perfusion regions of the four main branches of a coronary artery are an example of an anatomical region. The anatomical region dividing unit 15 is configured, for example, by using a learning model (segmentation model) trained by machine learning to perform a segmentation process that divides an input image into regions according to the perfusion regions of the four main branches and output the region-divided segmentation results.

[0101] The anatomical region information obtained by the anatomical region division unit 15 is sent to the clinical parameter calculation unit 16. The clinical parameter calculation unit 16 applies the anatomical region information to the pseudo ECG-gated CT image IMsyn generated by the image generation model 14, and calculates clinical parameters for each anatomical region. That is, in the case of the second embodiment, a CAC score is calculated for each perfusion region of the main branch. Other configurations may be the same as those described in FIG. 5.

[0102] Fig. 15 is a block diagram showing an example of a hardware configuration of an information processing device 100 according to the second embodiment. In the configuration shown in Fig. 15, elements that are the same as or similar to those in the configuration shown in Fig. 6 are given the same reference numerals, and duplicated explanations will be omitted.

[0103] The hardware configuration of the information processing device 100 may be the same as that of the information processing device 10 shown in FIG. 6. In addition to the configuration described in FIG. 6, an anatomical region division program 150 is stored in the computer-readable medium 104 of the information processing device 100. The anatomical region division program 150 may include a learned model trained by machine learning to perform segmentation processing that functions as the anatomical region division unit 15. The anatomical region division program 150 may also include an instruction to receive an input of an instruction to specify an anatomical region via a user interface, perform image division based on the specified information, and generate anatomical region information by identifying the anatomical region.

[0104] Moreover, the computer-readable medium 104 of the information processing device 100 includes an anatomical region information storage area 125. The anatomical region information storage area 125 stores anatomical region information obtained by the anatomical region division program 150. The clinical parameter calculation program 160 applies the anatomical region information to the generated image of the image generation model 14, and calculates clinical parameters for each anatomical region of the generated image individually. Other configurations may be similar to those described in FIG. 6.

[0105] 16 is an explanatory diagram showing an overview of a machine learning method for generating an image generation model 14 applied to the second embodiment. The training data set includes a non-ECG-gated CT image F16A obtained by imaging a subject, information on the perfusion regions of the four main branches of the coronary artery as anatomical region information for the non-ECG-gated CT image F16A, and a plurality of training data in which the correct CAC scores GTSs for each anatomical region calculated from an ECG-gated CT image F16B obtained by imaging the same subject as the non-ECG-gated CT image F16A are associated with each other.

[0106] The non-ECG-gated CT image F16A is a training image for input to the learning model 210. The correct CAC score GTSs is teacher data corresponding to the non-ECG-gated CT image F16A. The correct CAC score GTSs may be, for example, a value obtained by counting the number of voxels in an area having a CT value greater than 100 HU for each anatomical area in the ECG-gated CT image F16B. In the case of the second embodiment, the correct CAC score GTSs is a set of CAC scores for each perfusion area of ​​the main branch of the coronary artery. The anatomical area information may be generated from the non-ECG-gated CT image F16A or from the corresponding ECG-gated CT image F16B.

[0107] In this case, the learning model 210 is learned in the following procedure.

[0108] [Step 1] A non-ECG-gated CT image F16A is input to the learning model 210 as a training image.

[0109] [Step 2] A non-ECG-gated CT image F16A is input and a generated image F16C is output from the learning model 210.

[0110] [Step 3] Anatomical region information is applied to the generated image F16C to divide the generated image F16C into perfusion regions of the coronary artery branches, and threshold processing is performed on the generated image F16C under the condition of CT value > 100HU, and CAC scores PRSs are calculated for each perfusion region individually. The regional CAC scores PRSs calculated from the generated image F7C are CAC scores predicted from the non-ECG-gated CT image F16A. Figure 16 shows an example in which the values ​​of the regional CAC scores PRSs corresponding to the four main branches calculated from the generated image F7C are (RCA, LM, LAD, LCx) = (5, 2, 60, 45).

[0111] [Step 4] The machine learning device including a processor that executes machine learning calculates a loss from the CAC score PRSs calculated from the generated image F7C and the correct CAC score GTSs, and updates the model parameters of the learning model 210 so as to minimize the loss. FIG. 16 shows an example in which the values ​​of the correct CAC scores GTSs for each area corresponding to the four main branches are (RCA, LM, LAD, LCx)=(0, 0, 90, 30). The model parameters of the learning model 210 are updated so that the values ​​of each clinical parameter calculated from the generated image F7C for each anatomical area approach the values ​​of the correct clinical parameters for the corresponding anatomical area.

[0112] By repeating the above steps 1 to 4 using multiple training data, the model parameters of the learning model 210 are optimized, and the learning model 210 is trained to generate a generated image F16C for which a CAC score PRSs is calculated with the same accuracy as the correct CAC score GTSs.

[0113] Fig. 17 is a block diagram showing the functional configuration of a machine learning device 32 applied to the second embodiment. In the configuration shown in Fig. 17, elements that are the same as or similar to those in the configuration shown in Fig. 8 are given the same reference numerals, and duplicated explanations will be omitted.

[0114] The training data acquired by the machine learning device 32 includes anatomical region information AAj for a training image TIj that is a non-ECG-gated CT image. In addition, the correct clinical parameters associated with the training image TIj are correct clinical parameters GTsj for each anatomical region.

[0115] The clinical parameter calculation unit 220 calculates the clinical parameter PPsj for each anatomical region based on the generated image SIj and the anatomical region information AAj output from the learning model 210. Other configurations may be similar to those described in FIG.

[0116] Fig. 18 is a block diagram showing an example of a hardware configuration of a machine learning device 32 applied to the second embodiment. In the configuration shown in Fig. 18, elements that are the same as or similar to those in the configuration shown in Fig. 9 are given the same reference numerals, and duplicated explanations will be omitted.

[0117] The hardware configuration of the machine learning device 32 may be similar to the configuration shown in Fig. 9. In addition to the configuration described in Fig. 9, the computer-readable medium 304 stores an anatomical region segmentation program 450. The anatomical region segmentation program 450 may have a configuration similar to that of the anatomical region segmentation program 150 described in Fig. 15.

[0118] The training data storage unit 550 stores a training data set including a plurality of training data in which a training image TIj, anatomical region information AAj for the training image TIj, and correct clinical parameters GTsj for each anatomical region are associated with each other.

[0119] The data acquisition program 400 includes an instruction to execute a process of acquiring training data including anatomical region information AAj. The clinical parameter calculation program 420 includes an instruction to execute a process of calculating clinical parameters for each anatomical region from the generated image SIj based on the anatomical region information AAj and the generated image SIj. In addition, the loss calculation program 430 includes an instruction to execute a process of calculating a loss from clinical parameters PPsj for each anatomical region calculated from the generated image SIj and correct clinical parameters GTsj for each anatomical region linked to the training image TIj. Other configurations may be the same as the configurations described in FIG. 9.

[0120] Fig. 19 is a block diagram showing the functional configuration of a training data generation device 52 that performs processing to generate training data used in the machine learning method of the second embodiment. In Fig. 19, elements that are the same as or similar to the configuration shown in Fig. 10 are given the same reference numerals, and duplicated explanations will be omitted.

[0121] The training data generating device 52 has a configuration in which anatomical region dividing units 510A and 510B are added to the configuration shown in Fig. 10. Although the anatomical region dividing units 510A and 510B are illustrated in Fig. 19, the anatomical region dividing units 510A and 510B may be a common (single) processing unit. Furthermore, although Fig. 19 shows an example in which anatomical region information is acquired from each of the ECG-gated CT image GIj and the non-ECG-gated CT image NIj, the anatomical region information acquired from the non-ECG-gated CT image NIj by the anatomical region dividing unit 510B has approximately the same accuracy as the anatomical region information acquired from the ECG-gated CT image GIj, the anatomical region information may be acquired from only one of the images.

[0122] For example, the anatomical region information obtained from the ECG-gated CT image GIj may be associated with the training image TIj to be used as the anatomical region information AAj of the training data. Alternatively, the anatomical region information AAj obtained from the non-ECG-gated CT image NIj may be applied to the clinical parameter calculation unit 520 to calculate the clinical parameter CPsj for each anatomical region.

[0123] [Flowchart of the machine learning method executed by the machine learning device 32] Fig. 20 is a flowchart showing an example of a machine learning method executed by the machine learning device 30. In the flowchart shown in Fig. 20, steps common to those in Fig. 12 are given the same step numbers, and overlapping explanations will be omitted.

[0124] The flowchart shown in Fig. 20 includes step S107 instead of step S106 in Fig. 12. After step S104, in step S107, the processor 302 calculates a clinical parameter PPsj for each anatomical region from the pseudo image based on the pseudo image (generated image SIj) generated by the learning model and the anatomical region information AAj associated with the training image TIj.

[0125] In step S108, the processor 302 calculates a loss from the clinical parameters PPsj for each anatomical region calculated from the pseudo image and the ground-truth clinical parameters GTsj for each anatomical region associated with the training images TIj.

[0126] Other steps may be similar to those in the flowchart of Fig. 12. In this manner, the trained learning model 210 is incorporated into the information processing device 100 as the image generation model 14.

[0127] [Flowchart of information processing method executed by information processing device 100] Fig. 21 is a flowchart showing an example of an information processing method executed by the information processing device 100 according to the second embodiment. In the flowchart of Fig. 21, steps common to those in Fig. 13 are given the same step numbers, and overlapping descriptions will be omitted.

[0128] The flowchart shown in FIG. 21 includes steps S205 and S207 instead of step S206 in FIG.

[0129] After step S204, the processor 102 divides the target image into anatomical regions and obtains anatomical region information.

[0130] In step S207, the processor 102 calculates clinical parameters for each anatomical region from the pseudo image based on the pseudo image generated by the image generation model 14 and the anatomical region information. In the case of the second embodiment, the CAC score for each perfusion region of the four main branches of the coronary artery is calculated. The other steps may be the same as those in the flowchart of FIG.

[0131] [Verification of the accuracy of CAC scoring based on non-ECG-gated CT images] 22 is a graph showing the relationship between the CAC score calculated from the non-ECG-gated CT image using the information processing device 100 according to the second embodiment and the CAC score calculated from the corresponding ECG-gated CT image of the same subject. As shown in FIG. 22, the two CAC scores are roughly the same.

[0132] 23 is a graph showing a relationship between a CAC score calculated from a non-ECG-gated CT image using an information processing device according to a comparative example and a CAC score calculated from an ECG-gated CT image. The information processing device according to the comparative example is configured to calculate a CAC score by performing threshold processing on the non-ECG-gated CT image without using the image generation model 14. In this case, as shown in FIG. 23, the two CAC scores are significantly different from each other.

[0133] FIG. 24 shows an example of an image used in the CAC scoring shown in FIG. 22 and FIG. 23. The left image F24A in FIG. 24 is an example of an ECG-gated CT image. The center image F24B is an example of a non-ECG-gated CT image of the same subject as the left image F24A. The right image F24C is an example of a pseudo image generated by the image generation model 14 from the non-ECG-gated CT image of the center image F24B. As is clear from a comparison between the left image F24A and the right image F24C, the pseudo image output by the image generation model 14 can be an image having different appearance characteristics from the ECG-gated CT image of the actual image.

[0134] About the programs that run computers A program that causes a computer to realize some or all of the processing functions of each of the information processing device 10, the machine learning device 30, and the training data generation device 50 of the first embodiment, and the information processing device 100 of the second embodiment, the machine learning device 32, and the training data generation device 52 can be recorded on a computer-readable medium such as an optical disk, a magnetic disk, a semiconductor memory, or other tangible, non-transitory information storage medium, and the program can be provided through this information storage medium.

[0135] In addition, instead of providing the program by storing it on such a tangible, non-transitory computer-readable medium, it is also possible to provide the program signal as a download service using a telecommunications line such as the Internet.

[0136] Furthermore, some or all of the processing functions of each of the above-mentioned devices may be realized by cloud computing, and may also be provided as SaaS (Software as a Service).

[0137] <Hardware configuration of each processing unit> The hardware structure of the processing units that perform various processes, such as the data acquisition unit 12 in the information processing device 10 and the information processing device 100, the image generation unit including the image generation model 14, the anatomical region division unit 15, the clinical parameter calculation unit 16, the processing result output unit 18, the image generation unit including the learning model 210 in the machine learning device 30 and the machine learning device 32, the clinical parameter calculation unit 220, the loss calculation unit 230, the model parameter update amount calculation unit 232, the model parameter update unit 234, the anatomical region division units 510A, 510B, the clinical parameter calculation unit 520, and the association processing unit 530 in the training data generation device 50 and the training data generation device 52, is, for example, various processors as shown below.

[0138] Various types of processors include CPUs, which are general-purpose processors that execute programs and function as various processing units, GPUs, programmable logic devices (PLDs), such as FPGAs (Field Programmable Gate Arrays), which are processors whose circuit configuration can be changed after manufacture, and dedicated electrical circuits, such as ASICs (Application Specific Integrated Circuits), which are processors with a circuit configuration designed specifically to execute specific processes.

[0139] A processing unit may be composed of one of these various processors, or may be composed of two or more processors of the same type or different types. For example, a processing unit may be composed of multiple FPGAs, or a combination of a CPU and an FPGA, or a combination of a CPU and a GPU. Also, multiple processing units may be composed of one processor. As an example of multiple processing units being composed of one processor, first, as represented by a computer such as a client or a server, there is a form in which one processor is composed of a combination of one or more CPUs and software, and this processor functions as multiple processing units. Second, as represented by a system on chip (SoC), there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units in one IC (Integrated Circuit) chip. In this way, the various processing units are composed of one or more of the above various processors as a hardware structure.

[0140] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.

[0141] [Advantages of the first and second embodiments] According to the information processing device 10 according to the first embodiment and the information processing device 100 according to the second embodiment described above, the following effects can be obtained.

[0142] [1] It is possible to calculate CAC scores from non-ECG-gated CT images with the same accuracy as from ECG-gated CT images.

[0143] [2] Compared to ECG-gated CT images, non-ECG-gated CT images can be obtained using a simpler imaging method and do not require a particularly expensive, high-performance imaging system; they can be obtained using a standard CT scanner.

[0144] [3] Non-ECG-gated CT images involve less radiation exposure to the subject than ECG-gated CT images, and can be obtained with less stress on the subject.

[0145] [4] The calculation process of the CAC scores calculated by each of the information processing device 10 and the information processing device 100 is explainable, and the calculated values ​​are reasonably convincing.

[0146] Other clinical parameters: In the above-mentioned embodiments, the CAC score is calculated as a clinical parameter, but the clinical parameter is not limited to the CAC score and may be a CAC severity stage indicating the severity of calcification. In addition, the clinical parameter may be a calcium volume of a cardiovascular system or an Agatston score.

[0147] About types of medical images The technology of the present disclosure is not limited to CT images, but can be applied to various medical images captured by various medical devices (modalities), such as MR images captured by an MRI (Magnetic Resonance Imaging) device, ultrasound images projecting human body information, PET images captured by a Positron Emission Tomography (PET) device, and endoscopic images captured by an endoscopic device. Images targeted by the technology of the present disclosure are not limited to three-dimensional images, and may be two-dimensional images. In the case of a configuration that handles two-dimensional images, the word "voxel" in the content described in each of the above embodiments is replaced with "pixel" and applied.

[0148] "others" The present disclosure is not limited to the above-described embodiment, and various modifications are possible without departing from the spirit and scope of the technical idea of ​​the present disclosure. [Explanation of symbols]

[0149] 10, 100 Information processing device 12 Data Acquisition Section 14 Image Generation Model 15 Anatomical region division 16 Clinical parameter calculation section 18 Processing result output section 30, 32 Machine learning device 50, 52 Training data generation device 102 processors 104 Computer-readable medium 106 Communication Interface 108 Input / Output Interface 110 Bus 112 Memory 114 Storage 120 Input image storage area 122 Generated image storage area 124 Processing result storage area 125 Anatomical area information storage area 150 Anatomical Segmentation Program 152 Input Device 154 Display device 160 Clinical Parameter Calculation Program 180 Processing result output program 190 Display Control Program 210 Learning Model 220 Clinical Parameter Calculation Unit 230 Loss calculation section 232 Model parameter update amount calculation unit 234 Model Parameter Update Unit 302 Processor 304 Computer-readable medium 306 Communication Interface 308 Input / Output Interface 310 Bus 314 Storage 330 Learning Processing Program 340 Display Control Program 352 Input Device 354 Display device 400 Data Acquisition Program 420 Clinical Parameter Calculation Program 430 Loss Calculation Program 440 Optimizer 450 Anatomical Segmentation Program 510A Anatomical segmentation section 510B Anatomical segmentation section 520 Clinical Parameter Calculation Unit 530 Association processing unit 550 Training Data Storage Unit AAj Anatomical region information CPj Clinical parameters CPsj Clinical parameters F1A left diagram F1B right F2A Images F2B Images F3A ECG-synchronized CT images F4A Non-ECG-gated CT images F7A Non-ECG-gated CT images F7B ECG-synchronized CT images F7C generated image F16A Non-ECG-gated CT images F16B ECG-gated CT images F16C generated image F24A left diagram F24B center view F24C right GIj ECG-synchronized CT images GTj Correct Clinical Parameters GTS CAC Score GTsj Correct Clinical Parameters GTSs Correct CAC Score IMng Non-ECG-gated CT images IMsyn pseudo-ECG-gated CT images NIj Non-ECG-gated CT images PPj Clinical parameters PPsj Clinical parameters PRS CAC Score PRSs CAC score SIj generated image TIj training images S102~S112 Steps of Machine Learning Method S202-S208 Steps of information processing method

Claims

1. one or more processors; one or more storage devices that store programs to be executed by the one or more processors; the program includes an image generation model that is trained to generate, from an input first image, a second image that imitates an image obtained by an imaging protocol different from that of the first image; the image generation model is a model trained by machine learning using a plurality of training data in which training images captured by a first imaging protocol are associated with correct clinical parameters calculated from corresponding images captured by a second imaging protocol different from the first imaging protocol using the same type of modality as that used to capture the training images, so that clinical parameters calculated from generated images output by the image generation model in response to input of the training images approach the correct clinical parameters; The one or more processors: Accepting input of the first image captured according to the first imaging protocol; generating the second image from the first image using the image generation model; calculating a clinical parameter from the second image; Information processing device.

2. The one or more processors: Segmenting the first image into anatomical regions; calculating the clinical parameters for each of the divided anatomical regions; The information processing device according to claim 1 .

3. each of the training image and the corresponding image is divided into the anatomical regions, and the correct clinical parameter is calculated for each of the divided anatomical regions of the corresponding image; The image generation model is a model trained so that each of the clinical parameters calculated from the generated image for each of the divided anatomical regions of the training image approaches the correct clinical parameters for each of the anatomical regions of the corresponding image. The information processing device according to claim 2 .

4. The segmented anatomical region is a perfusion region of a coronary artery.

4. The information processing device according to claim 2 or 3.

5. The clinical parameter is calcium volume per major coronary artery branch; The information processing device according to claim 2 .

6. the first image is a non-ECG-gated CT image obtained by ECG-gated imaging, the second image is an image simulating an electrocardiogram-gated CT image obtained by electrocardiogram-gated imaging; The information processing device according to claim 1 .

7. The training images are taken under lower dose conditions than the corresponding images. The information processing device according to claim 1 .

8. The first imaging protocol has a lower dose than the second imaging protocol. The information processing device according to claim 1 .

9. The first imaging protocol has a slower scan speed than the second imaging protocol. The information processing device according to claim 1 .

10. The clinical parameter is a calcification score. The information processing device according to claim 1 .

11. The clinical parameter is cardiovascular calcium volume; The information processing device according to claim 1 .

12. The clinical parameter is the severity of coronary artery calcification. The information processing device according to claim 1 .

13. The image generation model is configured using a neural network.

2. The information processing device according to claim 1.

14. 1. An information processing method executed by one or more processors, comprising: the one or more processors: acquiring a first image captured according to a first imaging protocol; inputting the first image into an image generation model, and generating a second image from the first image using the image generation model, the second image simulating an image obtained when the same subject as that of the first image is photographed using a modality of the same type as that used to photograph the first image, and a second photographing protocol different from the first photographing protocol; calculating a clinical parameter from the second image; The image generation model is a model that is trained using a plurality of training data in which training images captured by the first imaging protocol and correct clinical parameters calculated from corresponding images captured by the second imaging protocol of the same subject as the training images using the same type of modality as that used to capture the training images are associated, and is trained so that clinical parameters calculated from generated images output by the image generation model in response to input of the training images approach the correct clinical parameters. Information processing methods.

15. On the computer, a function of acquiring a first image captured according to a first imaging protocol; a function of inputting the first image into an image generation model, and generating, from the first image using the image generation model, a second image that imitates an image obtained when the same subject as that of the first image is photographed using a modality of the same type as that used to photograph the first image, but using a second photographing protocol different from the first photographing protocol; calculating a clinical parameter from the second image; A program for realizing the above. The image generation model is a model trained by machine learning using a plurality of training data in which training images captured by the first imaging protocol and correct clinical parameters calculated from corresponding images captured by the second imaging protocol of the same subject as the training images using the same modality as that used to capture the training images are associated with each other, so that clinical parameters calculated from generated images output by the image generation model in response to input of the training images approach the correct clinical parameters. program.

16. A trained model that enables a computer to realize a function of generating a pseudo-image that imitates an image obtained from an input image using a different imaging protocol from the input image, A trained model that has been trained through machine learning using multiple training data in which training images taken using a first imaging protocol are associated with correct clinical parameters calculated from corresponding images taken of the same subject as the training images using the same type of modality as that used to take the training images using a second imaging protocol different from the first imaging protocol, so that the clinical parameters calculated from the generated images output by the training model in response to the input of the training images approach the correct clinical parameters.

17. A learning model generation method for generating a learning model that causes a computer to realize a function of generating a pseudo image from an input image that imitates an image obtained by a shooting protocol different from the input image, a system including one or more processors, performing machine learning using a plurality of training data in which training images captured by a first imaging protocol and correct clinical parameters calculated from corresponding images captured by a second imaging protocol different from the first imaging protocol using the same type of modality as that used to capture the training images are associated with each other; inputting the training images into a learning model, calculating clinical parameters from generated images output from the learning model, and training the learning model so that the clinical parameters calculated from the generated images approach the correct clinical parameters. Learning model generation method.

Images