Information processing system, information processing device, and information processing method

Abstract

To provide an information processing system, an information processing device, and an information processing method capable of executing accurate inference even for medical data as an inference object whose character is different from that of teacher data while securing a performance of an inference model.SOLUTION: An information processing device includes: a first inference unit that with first image data, which is one of CT image data acquired by using an energy integration type radiation detector and PCCT image data acquired by using a photon counting type radiation detector as an input, outputs second image data indicating the other image data in a pseudo manner; and a second inference unit learned by using the other image data for outputting medical information with the second image data output by the first inference unit as an input.SELECTED DRAWING: Figure 1

Description

Technical Field

[0006] , ,

[0005] , , ,

[0001] The present invention relates to an information processing system, an information processing apparatus, and an information processing method for making inferences on medical data to be inferred that have different properties from teacher data.

Background Art

[0002] Generally, the accuracy of an inference model based on machine learning depends on the accuracy and quantity of the teacher data used for learning the inference model. Also, due to overfitting or the like where the inference model fits too much to the teacher data, sufficient accuracy may not be exhibited for test data. Furthermore, when performing additional learning on an inference model that has been once subjected to learning processing using teacher data with new features, it is necessary to carefully observe the accuracy of the inference model before the additional learning and the accuracy of the inference model after the additional learning.

[0003] Here, in Prior Art Document 1, it is disclosed to perform additional learning of an inference model using newly generated teacher data.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the technique described in Patent Document 1, although it is disclosed to perform additional learning using newly generated teacher data, the possibility that the performance of the inference model may unintentionally deteriorate due to the additional learning is not considered.

[0006] The present invention aims to provide an information processing system, information processing device, and information processing method that can perform highly accurate inferences even for medical data that is different in nature from the training data, while ensuring the performance of the inference model. [Means for solving the problem]

[0007] To solve the above problems, an information processing apparatus according to one aspect of the present invention ,light PCCT image data obtained using a subcounting type radiation detector Ta As a first image data, CT image data obtained using an energy-integrating radiation detector. A first inference unit outputs a second image data that simulates the same, The first inference unit outputs the second image data, which is then used as input to output medical information. CT It comprises a second inference unit trained using image data. [Effects of the Invention]

[0008] According to the present invention, while ensuring the performance of the inference model, it is possible to perform highly accurate inferences even for medical data that has different properties from the training data. [Brief explanation of the drawing]

[0009] [Figure 1] A diagram showing an example of the functional configuration of an information processing system according to the first embodiment. [Figure 2] A diagram showing an example of the hardware configuration of the information processing device of the present invention. [Figure 3] A diagram showing an example of the processing flow of an information processing system according to the first embodiment. [Figure 4] A schematic diagram showing the relationship between the inference unit and the storage unit of the information processing system according to the first embodiment. [Figure 5] A schematic diagram illustrating an example of a method for generating a trained model according to the first embodiment. [Figure 6] A schematic diagram showing an example of the inference processing of the inference unit according to the first embodiment. [Figure 7]A diagram schematically showing Modification 1 according to the first embodiment. [Figure 8] A diagram showing an example of the functional configuration of an information processing system according to the second embodiment. [Figure 9] A diagram showing an example of the processing flow of an information processing system according to the second embodiment [Figure 10] A diagram schematically showing an example of a method for generating a learned model according to the second embodiment. [Figure 11] A diagram schematically showing an example of the inference process of an inference unit according to the second embodiment. [Figure 12] A diagram schematically showing Modification 1 according to the second embodiment. [Figure 13] A diagram showing an example of the configuration of the mechanism of Modification 2 according to the second embodiment. [Figure 14] A diagram showing an example of the processing flow of Modification 2 according to the second embodiment.

MODE FOR CARRYING OUT THE INVENTION

[0010] <000…>The present invention can be preferably applied to, for example, sinogram data acquired by modality, medical data generated by image data reconstruction from raw data, and the like.

[0011] Modality is applicable to, for example, CT data and PCCT data acquired by an X-ray CT apparatus, a photon counting CT (PCCT: Photon Counting Computed Tomography), etc., and CT image data and PCCT image data generated by image data reconstruction from the data.

[0012] Hereinafter, an information processing apparatus, an information processing system, and an information processing method will be described with reference to the drawings. Note that the following embodiments are examples, and the information processing apparatus, information processing system, and information processing method according to the present application are not limited to the following embodiments.

[0013] (First Embodiment) Here, with reference to FIG. 1, an information processing system including the information processing apparatus of the present invention will be described. The information processing system in the present invention includes an information processing apparatus 1, a data server 2, a display device 3, and an input device 4. Note that the information processing apparatus 1 and the data server 2 are configured to be connectable via a network.

[0014] The information processing apparatus 1 includes an acquisition unit 10 that acquires medical data to be inferred, an inference unit 11 that executes a predetermined inference process on the acquired medical data to be inferred, and a display control unit 12 that causes the inference result to be displayed on the display device 3. The inference unit 11 includes a first inference unit 101 and a second inference unit 102.

[0015] The first inference unit 101 uses a first learned model based on machine learning or deep learning such as GAN (Generative Adversarial Network) to infer, from the first medical image data, which is the medical data to be inferred, pseudo second medical image data that is different from the first medical image data. Here, the pseudo second medical image data is input data for a second learned model described later. The second learned model is an inference model that has been subjected to learning processing so as to perform a predetermined inference on the input data.

[0016] The second inference unit 102 performs a predetermined inference process on the pseudo second medical image data output from the first inference unit 101 using the second learned model.

[0017] By being configured in this way, even when the medical data to be inferred is data having a different nature from the teacher data used for the learning of the second learned model that performs a predetermined inference process, the information processing apparatus 1 can perform an inference process using the first learned model that infers pseudo second data, and thus can perform a highly accurate inference using the second learned model.

[0018] Data server 2 stores medical data transmitted from various modalities. It may also store information about trained models in association with supplementary information. The trained models are downloaded and used by information processing device 1. Alternatively, trained models generated by performing a separate training process within information processing device 1 may be managed by the data server, or the information about the trained models may be stored in a storage unit (not shown) within information processing device 1.

[0019] Display device 3 is a display attached to information processing device 1, or a mobile terminal used by hospital staff via an external server.

[0020] Input device 4 is a mouse, keyboard, etc.

[0021] Figure 2 shows an example of the specific configuration of the information processing device 1. In this example, the information processing device 1 has a CPU 20, a GPU 21, RAM 22, ROM 23, and a storage device 24, which are connected by a system bus 25. The information processing device 1 is also connected to a display device 3 and an input device 4 such as a mouse or keyboard.

[0022] The flow of the inference processing performed by the information processing device 1 in this embodiment will be explained below with reference to Figure 3. In this embodiment, PCCT image data of a subject is used as the medical data to be inferred. The first trained model takes the PCCT image data as input and infers pseudo-CT image data. The second trained model then takes the inferred pseudo-CT image data as input and performs predetermined inference. PCCT image data is image data generated by reconstructing PCCT data obtained using a photon counting type radiation detector using known reconstruction techniques. CT image data is image data generated by reconstructing CT data obtained using an energy integrating type radiation detector. Note that energy integrating type radiation detectors and photon counting type radiation detectors are different types of radiation detectors. There is a large amount of CT image data accumulated to date, and by using this large amount of data to train the inference model, it is possible to perform highly accurate inference processing on CT image data. On the other hand, while PCCT image data can be acquired with high resolution, the number of data points is often smaller than that of CT image data. Because CT and PCCT scanners, which acquire CT and PCCT image data, are extremely expensive, it may be possible to acquire CT image data, which was more widely adopted in the time series, or to generate trained models using CT image data, but not be able to acquire PCCT image data. Alternatively, even if a similar amount of data can be acquired, creating the correct label information (generated through diagnosis or image interpretation) for the data is a very time-consuming process, and it is possible that the amount of training data for PCCT image data is less than the amount of training data for CT image data.

[0023] A trained model that infers PCCT image data from CT image data is created based on training data collected from institutions that possess the necessary PCCT image data, such as PCCT imaging equipment. Training data is generated, for example, by associating CT image data and PCCT image data taken of the same subject. Furthermore, since image data generation models such as GANs use pairs of image data as training data, the time-consuming task of adding label information is unnecessary, resulting in a low cost for generating a trained model.

[0024] Here, the predetermined inference processing performed by the second trained model refers to outputting medical information that is the result of performing inference tasks such as classification, detection, and extraction on a predetermined target (at least one of the following: an organ, a lesion region, or the degree of malignancy of the lesion).

[0025] In step S30, the acquisition unit 10 acquires PCCT image data as medical data to be inferred, transmits the acquired PCCT image data to the inference unit 11, and proceeds to step S31. The acquisition unit 10 may also acquire the type of image data from DICOM information etc. associated with the medical data to be inferred and transmit it to the inference unit 11. In any of the processes from step S31 to step S33, known preprocessing techniques may be applied to the image data. For example, processing the acquired image data to match the input size of the trained model is one such example.

[0026] In step S31, the inference unit 11 obtains data for a trained model used to perform inference processing on the medical data to be inferred, from the data server 2 or a storage unit provided in the information processing device 1.

[0027] The acquisition of the trained model by the inference unit 11 will be explained using Figure 4. The inference unit 11 acquires trained model data from the data server 2. The data server 2 stores associated information with the trained model, and this associated information includes, for example, information about the inference target corresponding to the trained model, and information about the training data used to generate the trained model. The inference unit 11 may acquire the trained model data based on the associated information and information about the type of image data transmitted from the acquisition unit 10. In this embodiment, as an example, PCCT image data is assumed to have been acquired as the medical data to be inferred.

[0028] Here, the generation process for the first and second trained models in this embodiment will be explained using Figure 5.

[0029] Figure 5(a) shows the training process for generating the first pre-trained model. The first pre-trained model is an image data generation model such as a GAN, and is generated by training it using machine learning on pairs of input and output image data. The first pre-trained model is generated by machine learning on an image data generation model such as a GAN based on training data consisting of pairs of CT image data and PCCT image data.

[0030] Figure 5(b) shows the training process for generating the second pre-trained model. The second pre-trained model is an inference model based on machine learning such as CNN, Transformer extended for image data processing, support vector machine, or random forest, and is generated by performing a training process using training data consisting of pairs of image data and corresponding ground truth data. In this embodiment, the image data is CT image data.

[0031] The inference unit 11 transmits model data of a first trained model that infers pseudo-CT image data from PCCT image data to the first inference unit 101. Furthermore, it transmits model data of a second trained model that performs predetermined inference processing from CT image data to the second inference unit 102, and proceeds to step S32. Steps S32 and S33, described later, may be omitted as appropriate depending on the type of image data. For example, if CT image data is acquired as the medical data to be inferred, step S32 may be omitted and the process may proceed to step S33. In that case, the inference processing performed in step S33 does not have to be pseudo-CT image data. That is, the inference unit 11 further acquires the type of image data to be inferred and the type of image data used for training the second inference unit 102, and if the acquired types are different, it inputs the output of the first inference unit 101 to the second inference unit 102. Furthermore, if the acquired types are the same, the inference unit 11 inputs the image data to be inferred to the second inference unit 102 without going through the first inference unit 101. In other words, the inference unit 11 acquires the type of input image data used for training the inference unit, and based on the type, selectively inputs to the inference unit one of the CT image data generated based on the integration of energy detected by the radiation detector and the PCCT image data generated based on the count of photons detected by the radiation detector.

[0032] In step S32, the first inference unit 101 performs a first inference process using the first trained model to infer pseudo-CT image data from the PCCT image data, which is the medical data to be inferred. The generated pseudo-CT image data is then sent to the second inference unit 102, and the process proceeds to step S33.

[0033] In step S33, the second inference unit 102 performs a second inference process on pseudo-CT image data using a second trained model that performs predetermined inferences from CT image data. Here, the second trained model is a trained model generated from a wealth of previously accumulated training data, and can perform highly accurate inference processing on CT image data. When the inference processing by the trained model is completed, the inference unit 11 transmits the inference result to the display control unit 12 and proceeds to step S34. The information processing device 1 may use a dataset that associates the results of the inference processing with PCCT image data as a training dataset for training the inference model. It may also have a learning unit for training the inference model.

[0034] Here, the flow of the inference processing in steps S32 and S33 according to this embodiment will be explained using the schematic diagram in Figure 6. When PCCT image data (original) is acquired as the image data to be inferred, the first inference unit 101 inputs the PCCT image data to the first trained model and generates pseudo-CT image data through the first inference process. Furthermore, the second inference unit 102 takes the pseudo-CT image data generated by the first inference process as input to the second trained model and outputs a predetermined inference result.

[0035] In step S34, the display control unit 12 causes the display device 3, such as a display, to display the inference results from the inference unit 11. The display control unit 12 may also display the medical data to be inferred, the pseudo-image data generated from the medical data to be inferred by the first inference process, and the predetermined inference results obtained by the second inference process in a comparable manner.

[0036] In this embodiment, the information processing device 1, configured as described above, can perform highly accurate inferences even for medical data that is different in nature from the training data, while ensuring the performance of the inference model.

[0037] (Variation 1) In the above-described embodiment, the first inference unit 101 applies a first inference process to PCCT image data to generate pseudo-CT image data. Then, the second inference unit 102 applies a second inference process to the pseudo-CT image data to perform a predetermined inference process, and the display control unit 12 displays the inference result on the display device 3.

[0038] In this modified version, the inference unit 11 further reflects the inference results from the second inference unit 102 into the original image data to be inferred. For example, the second inference unit 102 may associate the region extracted by inference with the original image data to be inferred and display it on the display device 3. The extracted region may be, for example, an organ region or a lesion region.

[0039] Furthermore, the inference unit 11 uses known technologies such as Grad-CAM to calculate the regions in the image data that contributed to the calculation of the inference result during the inference process. In addition, by displaying information that reflects the pixel information that contributed to a predetermined inference process in the original image data to be inferred through the second inference process, the user can understand which regions of the original image data to be inferred contributed to the calculation of the inference result.

[0040] Figure 7 is a schematic diagram of the processing in this modified example. The second inference unit 102 calculates the inference result using Grad-CAM when performing a second inference process using the second trained model, and reflects the influencing pixel information in the pseudo-CT image data input to the second trained model. Furthermore, the influence map for the pseudo-CT image data is mapped to the original data, and the display control unit 12 displays it on the display device 3. In this modified example, it can be determined from both the inference target image data and the pseudo-CT image data that the left lung influenced the calculation of the inference result. This mapping can be done simply by reflecting the coordinate information in the original image data, provided that the positional relationship between the output image data and the input image data is maintained using a GAN technology such as Pix2Pix. Alternatively, feature points can be calculated using other inference models or known technologies, and the deformation information due to the alignment of image data can be reflected in the influence map.

[0041] (Second embodiment) In the first embodiment, an information processing system / information processing device was described in which a first inference unit uses a first trained model to infer CT image data as pseudo-second data from PCCT image data, which is medical data to be inferred.

[0042] In this embodiment, the information processing system including the information processing device 8 is a system in which a first inference unit infers pseudo-PCCT image data from CT image data, and a second inference unit performs predetermined inferences from the pseudo-PCCT image data.

[0043] This configuration allows for the application of predetermined inference processing to PCCT image data, which is higher resolution image data than CT data obtained using conventional energy-integrating radiation detectors, thereby enabling the acquisition of highly accurate inference results.

[0044] The information processing system of this embodiment will be described below with reference to Figure 8. Note that components similar to those in the first embodiment will be given the same figure numbers, and explanations will be omitted where appropriate. The information processing system in this embodiment comprises an information processing device 8, a data server 2, a display device 3, and an input device 4.

[0045] The information processing device 8 includes an acquisition unit 80 for acquiring medical data to be inferred, an inference unit 81 for executing predetermined inference processing on the acquired medical data to be inferred, and a display control unit 82 for displaying the inference results on the display device 3. The inference unit 81 includes a first inference unit 801 and a second inference unit 802.

[0046] The first inference unit 801 infers pseudo-PCCT image data from CT image data, which is the medical data to be inferred. The first inference unit 801 performs inference processing on the medical data to be inferred using a first pre-trained model based on machine learning such as GAN (Generative Adversarial Network) or deep learning. Here, the pseudo-PCCT image data inferred by the first inference unit 801 is input data to the second pre-trained model used by the second inference unit 802. The second pre-trained model is an inference model that has been trained to perform predetermined inferences on the input data.

[0047] The second inference unit 802 performs predetermined inference processing on the pseudo-PCCT image data output from the first inference unit 801 using the second trained model. Here, the predetermined inference processing performed by the second trained model refers to, for example, performing inference tasks such as classification, detection, and extraction on predetermined targets (at least one of organs, lesion regions, or malignancy levels of lesions).

[0048] The flow of the inference process performed by the information processing device 8 in this embodiment will be explained below with reference to Figure 9. In this embodiment, CT image data of a subject is used as the medical data to be inferred. The first trained model takes the CT image data as input and infers pseudo-PCCT image data, and the second trained model takes the inferred pseudo-PCCT image data as input and performs predetermined inference. PCCT image data is image data with higher resolution than CT image data, generated by reconstructing PCCT data obtained using a photon counting type radiation detector using known reconstruction techniques. CT image data is image data generated by reconstructing CT data obtained using an energy integrating type radiation detector.

[0049] Because PCCT equipment, which is used to acquire PCCT image data, is very expensive, there may be cases where a CT scanner is installed but a PCCT scanner is not. In such cases, it is possible to infer pseudo-PCCT image data, perform predetermined inferences on the PCCT image data, and, for example, if a malignant tumor is detected, display information recommending PCCT imaging to the patient can be shown.

[0050] In step S80, the acquisition unit 80 acquires CT image data from the data server 2 as medical data to be inferred, transmits the acquired data to the inference unit 81, and proceeds to step S81. The acquisition unit 80 may also acquire the type of image data from DICOM information etc. attached to the medical data to be inferred and transmit it to the inference unit 81. In any of the processes from step S81 to step S83, known preprocessing techniques may be applied to the image data. For example, processing the acquired image data to match the input size of the trained model is one such example.

[0051] In step S81, the inference unit 81 acquires data of a trained model for inference processing from the data server 2 or a storage unit provided in the information processing device 8. The data server 2 stores, for example, associated supplementary information with the trained model, and this supplementary information includes, for example, information about the inference target corresponding to the trained model, and information about the training data used to generate the trained model. The inference unit 81 may acquire model data of the trained model based on the supplementary information and information about the type of image data transmitted from the acquisition unit 80.

[0052] Here, the generation process for the first and second trained models in this embodiment will be explained using Figure 10.

[0053] Figure 10(a) shows the training process for generating the first pre-trained model. The first pre-trained model is an image data generation model such as a GAN, and is generated by training it using machine learning on pairs of input and output image data. The first pre-trained model is generated by machine learning on an image data generation model such as a GAN, based on training data consisting of pairs of PCCT image data and CT image data.

[0054] Figure 10(b) shows the training process for generating a second pre-trained model. The second pre-trained model is an inference model based on machine learning such as CNN, Transformer extended for image data processing, support vector machine, or random forest, and is generated by performing a training process using training data consisting of pairs of image data and corresponding ground truth data. In this embodiment, the image data is PCCT image data.

[0055] The inference unit 81 transmits model data of a first trained model that infers pseudo-PCCT image data from CT image data to the first inference unit 801. Furthermore, it transmits model data of a second trained model that performs predetermined inference processing from the pseudo-PCCT image data to the second inference unit 102, and proceeds to step S82.

[0056] Steps S82 and S83, described later, may be omitted as appropriate depending on the type of image data. For example, if PCCT image data is acquired as the medical data to be inferred, step S82 may be omitted and the process may proceed to step S83. In that case, the inference processing performed in step S83 does not need to be performed on pseudo-PCCT image data. That is, the inference unit 81 further acquires the type of image data to be inferred and the type of image data used for training the second inference unit, and if the acquired types are different, it inputs the output of the first inference unit 801 to the second inference unit 802. On the other hand, if the acquired types are the same, the inference unit 81 inputs the image data to be inferred to the second inference unit without going through the first inference unit 801. In other words, the inference unit 81 acquires the type of input image data used for training the inference unit, and based on the type, selectively inputs one of the CT image data generated based on the integration of energy detected by the radiation detector and the PCCT image data generated based on the count of photons detected by the radiation detector to the inference unit.

[0057] In step S82, the first inference unit 801 performs a first inference process using the first trained model to infer pseudo-PCCT image data from the CT image data, which is the medical data to be inferred. The generated pseudo-PCCT image data is then sent to the second inference unit 102, and the process proceeds to step S83.

[0058] In step S83, the second inference unit 802 performs a second inference process on pseudo-CT image data using a second trained model that performs predetermined inferences from PCCT image data. Here, the second trained model is a trained model generated from a wealth of training data accumulated in the past, and can perform highly accurate inference processing on CT image data. When the inference processing by the trained model is completed, the inference unit 11 transmits the inference result to the display control unit 12 and proceeds to step S84.

[0059] Here, using Figure 11, the flow of the inference processing in steps S82 and S83 according to this embodiment will be explained. When CT image data (original) is acquired as the image data to be inferred, the first inference unit 801 inputs the CT image data to the first trained model and generates pseudo-PCCT image data through the first inference process. Furthermore, the second inference unit 102 takes the pseudo-PCCT image data generated by the first inference process as input to the second trained model and outputs a predetermined inference result.

[0060] In step S84, the display control unit 82 causes the display device 3, such as a display, to display the inference results from the inference unit 81. The display control unit 82 may also display the medical data to be inferred, the pseudo-image data generated from the medical data to be inferred by the first inference process, and the predetermined inference results obtained by the second inference process in a comparable manner.

[0061] In this embodiment, the information processing device 8, configured as described above, can perform highly accurate inferences even for medical data that is different in nature from the training data, while ensuring the performance of the inference model.

[0062] As described in the first and second embodiments, the information processing device of the present invention includes: a first inference unit that takes first image data, which is one of the image data obtained using an energy-integrating type radiation detector or PCCT image data obtained using a photon-counting type radiation detector, as input and outputs second image data that pseudo-represents the other image data; and a second inference unit that takes the second image data output by the first inference unit as input and outputs medical information. The inference target is not limited to medical image data. For example, processing may be performed using CT data or PCCT data, which are medical data before reconstruction, instead of the image data of the embodiments described above. In other words, the scope of the present invention also includes an information processing device that includes: a first inference unit that takes first data, which is one of the data obtained using an energy-integrating type radiation detector or PCCT data obtained using a photon-counting type radiation detector, as input and outputs second data that pseudo-represents the other data; and a second inference unit that takes the second data generated by the first inference unit as input and outputs medical information.

[0063] (Modification 2) In this modified version, the inference unit 81 further uses known technologies such as Grad-CAM to calculate the regions in the image data that contributed to the calculation of the inference result during the inference process. Furthermore, by reflecting the pixel information that contributed to the predetermined inference process in the original image data to be inferred through the second inference process, the user can understand which regions of the original image data to be inferred contributed to the calculation of the inference result.

[0064] Figure 12 is a schematic diagram of the processing in this modified example. The second inference unit 102 calculates the inference result using Grad-CAM when performing a second inference process using the second trained model, and reflects the influencing pixel information in the pseudo-PCCT image data input to the second trained model. Furthermore, the influence map for the pseudo-PCCT image data is mapped to the original data, and the display control unit 82 displays it on the display device 3. In this modified example, it can be determined from both the inference target image data and the pseudo-CT image data that the left lung influenced the calculation of the inference result. This mapping can be done simply by reflecting the coordinate information in the original image data, provided that the positional relationship between the output image data and the input image data is maintained using a GAN technology such as Pix2Pix. Alternatively, feature points can be calculated using other inference models or known technologies, and the deformation information due to the alignment between image data can be reflected in the influence map.

[0065] (Variation 3) This modified example will be explained using Figure 13. In this modified example, the information processing device 8 further includes a determination unit 1301 that determines whether the inference result from the inference unit 81 satisfies predetermined conditions. If the predetermined inference result from the inference unit 81 satisfies predetermined conditions, the determination unit 1301 causes the display device 3, via the display control unit 1302, to perform a display process recommending the acquisition of PCCT image data. Specifically, if a tumor with high urgency or malignancy is detected in the simulated PCCT image data, the display control unit 1302 displays information recommending the acquisition of PCCT image data.

[0066] The processing flow of this modified example will be explained using Figure 14.

[0067] Steps S80 to S83 are the same as in the second embodiment, so their explanation will be omitted.

[0068] In step S1401, the determination unit 1301 determines whether the predetermined inference result from the inference unit 81 satisfies predetermined conditions. The determination unit 1301 makes this determination, for example, by comparing the likelihood of the inference result with predetermined threshold values. The determination unit 1301 applies a first threshold to the likelihood, which indicates the probability that the lesion is malignant, as the inference result from the inference unit 81. If the likelihood is greater than the threshold, or if the likelihood is greater than the second threshold but less than the third threshold, the process proceeds to step S1402. On the other hand, if the determination unit 1301 determines that the predetermined inference result from the inference unit 81 does not satisfy predetermined conditions, the process proceeds to step S84.

[0069] In step S1402, the display control unit 1302 causes the display device 3 to display information recommending the acquisition of PCCT image data by the PCCT device.

[0070] This modified version, when configured in this way, allows for the inference of pseudo-PCCT image data from CT image data for a subject from whom CT image data has been taken. If a predetermined inference process is performed on the pseudo-PCCT image data with high accuracy, and the degree of malignancy or urgency of the lesion is high, then display information recommending that the subject take actual PCCT image data can be displayed.

[0071] (Other examples) Furthermore, the present invention can also be realized by performing the following process: that is, supplying software (program) that realizes the functions of the above-described embodiment to a system or device via a network or various storage media, and having the computer (or CPU or MPU, etc.) of that system or device read and execute the program.

Claims

1. A first inference unit that takes first image data, which is PCCT image data obtained using a photon counting type radiation detector, as input and outputs second image data that simulates CT image data obtained using an energy integrating type radiation detector, A second inference unit, trained using the CT image data, takes the second image data output by the first inference unit as input and outputs medical information. An information processing device equipped with the following features.

2. The information processing device further acquires pixel information that contributed to the inference of the medical information, A display control unit that displays the pixel information on a display means in association with the first image data, The information processing apparatus according to claim 1, further comprising:

3. The medical information includes the region extracted from the second image data. The information processing apparatus according to claim 1, further comprising a display control unit that causes the information processing apparatus to display the region on a display means in association with the first image data.

4. The information processing apparatus according to claim 3, wherein the extracted region includes at least one of an organ region and a lesion region.

5. The information processing apparatus according to claim 1, further comprising a learning unit that learns an inference model using a dataset of pairs of PCCT image data and medical information output by the second inference unit.

6. The information processing device further acquires the type of image data to be inferred and the type of input image data used for training the second inference unit. The information processing apparatus according to any one of claims 1 to 4, wherein if the acquired types are different, the output of the first inference unit is input to the second inference unit.

7. The information processing apparatus according to claim 6, wherein if the acquired types are the same, the image data to be inferred is input to the second inference unit without going through the first inference unit.

8. The information processing apparatus according to any one of claims 1 to 4, wherein the radiation detector is different from the energy-integrating type radiation detector and the photon-counting type radiation detector.

9. A first inference unit that takes first data, which is PCCT data obtained using a photon counting type radiation detector, as input and outputs second data that pseudo-represents CT data obtained using an energy integrating type radiation detector, A second inference unit, trained using the CT data, takes the second data generated by the first inference unit as input and outputs medical information. An information processing device equipped with the following features.

10. A first inference unit takes first image data, which is PCCT image data obtained using a photon counting type radiation detector, as input and outputs second image data that simulates CT image data obtained using an energy integrating type radiation detector, and inputs the first image data, An information processing method for obtaining medical information from a second inference unit that has been trained using CT image data, which takes the second image data output by the first inference unit as input and outputs medical information.

11. A first inference unit that takes first image data, which is PCCT image data obtained using a photon counting type radiation detector, as input and outputs second image data that simulates CT image data obtained using an energy integrating type radiation detector, A second inference unit, trained using the CT image data, takes the second image data output by the first inference unit as input and outputs medical information. An information processing system equipped with the following features.

Images