Information processing device, information processing method, and program

Abstract

This information processing device 1 comprises: a storage unit 12 which stores a prognosis estimation model that has been trained to output the prognosis of a patient upon receiving input data including a spatial distribution of at least one among prescribed proteins and feature amounts pertaining to prescribed biomarkers in specimens collected from the patient; an acquisition unit 131 which acquires a spatial distribution of at least one among prescribed proteins and feature amounts pertaining to prescribed biomarkers in the specimens collected from the target patient; and a prognosis estimation unit 132 which outputs, as the estimated values of the prognosis of the target patient, information output by inputting, to the prognosis estimation model, input data including the spatial distribution acquired by the acquisition unit 131.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] It is known that CD4-positive and CD8-positive T cell lymphocytes particularly affect the prognosis of cancer (for example, see Non-Patent Document 1), and research has been conducted on prognostic prediction using these.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the prior art, there has been a problem that the information available for prognostic prediction is limited, so the improvement of prediction accuracy is also limited.

[0005] Therefore, the present invention has been made in view of these points, and an object thereof is to provide a method for improving the accuracy of prognostic prediction of diseases.

Means for Solving the Problems

[0006] An information processing apparatus according to a first aspect of the present invention includes: a storage unit that stores a prognosis estimation model that has been trained to output the prognosis of a patient when input data including a feature quantity relating to a predetermined biomarker and the spatial distribution of at least one of a predetermined protein in a sample taken from a patient is input; an acquisition unit that acquires the feature quantity relating to a predetermined biomarker and the spatial distribution of at least one of a predetermined protein in a sample taken from a target patient; and a prognosis estimation unit that outputs the information output by inputting the input data including the spatial distribution acquired by the acquisition unit into the prognosis estimation model as an estimated value of the prognosis of the target patient.

[0007] The memory unit stores the prognosis estimation model, which has been trained using drugs administered to the patient as further input; the acquisition unit acquires drugs to be administered to the target patient; and the prognosis estimation unit inputs the drugs to be administered to the target patient acquired by the acquisition unit into the prognosis estimation model, and the output information may be output as an estimated value of the prognosis of the target patient.

[0008] The spatial distribution of the features relating to the predetermined biomarker may be the spatial distribution of features relating to high-frequency microsatellite instability or BRAF gene mutations in the sample taken from the patient.

[0009] The storage unit further stores a distribution estimation model that has been trained to output the spatial distribution of features relating to a predetermined biomarker in the image data when it receives image data of a sample, the acquisition unit acquires image data of a sample taken from the target patient, and the information processing device may further include a distribution generation unit that inputs the image data of the sample acquired by the acquisition unit into the distribution estimation model and generates the spatial distribution.

[0010] The spatial distribution of the predetermined protein may be the spatial distribution of tumor tissue and CD3-positive lymphocytes or CD20-positive lymphocytes in the sample taken from the patient.

[0011] The memory unit stores the prognosis estimation model, which has been learned by further inputting the spatial distribution of tumor tissue and predetermined proteins in a sample taken from a patient; the acquisition unit further acquires the spatial distribution of tumor tissue and predetermined proteins in a sample taken from the target patient; and the prognosis estimation unit may output the information output by inputting the input data, which further includes the spatial distribution of tumor tissue and predetermined proteins acquired by the acquisition unit, into the prognosis estimation model, as an estimated prognosis for the target patient.

[0012] The acquisition unit acquires first sample image data, which is image data of a sample taken from the target patient, which has been processed in a predetermined way to detect the composition of cells or tissues in the sample, and second sample image data, which is image data of the sample that has been processed in a predetermined way to detect a predetermined protein in the sample. The information processing device may further include a distribution generation unit that generates a spatial distribution of tumor tissue and a predetermined protein in the sample based on the first sample image data and the second sample image data acquired by the acquisition unit.

[0013] The first sample image data and the second sample image data are image data obtained by staining a sample taken from the patient in different ways. The information processing device further includes a registration unit that associates a position in the first sample image data with a position in the second sample image data. The distribution generation unit may generate a spatial distribution of tumor tissue and a predetermined protein in the sample based on the first sample image data and the second sample image data associated by the registration unit.

[0014] The memory unit stores the prognosis estimation model, which has been learned by further inputting the spatial distribution of features related to tumor tissue in a sample taken from a patient; the acquisition unit further acquires the spatial distribution of features related to tumor tissue in a sample taken from the target patient; and the prognosis estimation unit may output the information output by inputting the input data, which further includes the spatial distribution of features related to tumor tissue acquired by the acquisition unit, into the prognosis estimation model, as an estimated value of the prognosis for the target patient.

[0015] A second aspect of the present invention relates to an information processing method which includes the steps of: a computer performing the steps of acquiring the spatial distribution of at least one of a predetermined biomarker feature quantity and a predetermined protein in a sample taken from a target patient; and inputting the input data, including the spatial distribution acquired in the acquisition step, into a prognosis estimation model stored in a storage unit, and outputting the resulting information as an estimated prognosis for the target patient.

[0016] In a program according to a third aspect of the present invention, the computer is made to perform the steps of: acquiring the spatial distribution of at least one of a predetermined biomarker feature quantity and a predetermined protein in a sample taken from a target patient; and inputting the input data, including the spatial distribution acquired in the acquisition step, into a prognosis estimation model stored in a memory unit, and outputting the output information as an estimated prognosis for the target patient. [Effects of the Invention]

[0017] The present invention provides a method for improving the accuracy of predicting disease prognosis. [Brief explanation of the drawing]

[0018] [Figure 1] This is a diagram illustrating the overview of the processing in the information processing device 1 according to the embodiment. [Figure 2] This is a block diagram showing the configuration of the information processing device 1. [Figure 3]It is a diagram showing an example of the processing in the distribution generation unit 133. [Figure 4] It is a diagram showing an example of the processing in the distribution generation unit 133. [Figure 5] It is a flowchart showing the processing flow in the information processing apparatus 1. [Figure 6] It is a diagram for explaining the outline of the processing in the information processing apparatus 1 according to the first modification. [Figure 7] It is a diagram for explaining the outline of the processing in the information processing apparatus 1 according to the second modification. [Embodiment for Carrying out the Invention]

[0019] [Outline of Information Processing Apparatus 1] FIG. 1 is a diagram for explaining the outline of the processing in the information processing apparatus 1 according to the embodiment. The information processing apparatus 1 is an apparatus for estimating the prognosis of a patient to be determined based on the spatial distribution of a predetermined index in a specimen collected from the patient. The information processing apparatus 1 is, for example, a server, a personal computer, or the like.

[0020] The information processing apparatus 1 inputs input information including the spatial distribution D1 into the prognosis estimation model M1 and outputs a prognosis estimation value D2. The spatial distribution D1 is information that spatially shows the degree to which a feature amount related to a predetermined biomarker, a tumor tissue, or a predetermined type of lymphocyte (protein), etc. is distributed in the image data obtained by imaging a specimen collected from the patient to be estimated.

[0021] The prognosis estimation model M1 is a learned model that has learned the spatial distributions of biomarkers, tumor tissues, predetermined types of lymphocytes, etc. in the specimen as teacher data. When the spatial distribution D1 of the patient to be determined is input to the prognosis estimation model M1, the prognosis estimation value D2 is output. The prognosis estimation model M1 may output the prognosis estimation value D2 based on the spatial distributions of a plurality of indexes, or may output the prognosis estimation value D2 based on other input data in addition to the spatial distribution.

[0022] The prognosis estimate D2 is an estimate of the prognosis of the patient being estimated. As an example, the prognosis estimate D2 indicates whether the probability of survival for a predetermined period from the time the sample was obtained is above a predetermined threshold. In Figure 1, an example is shown in which the prognosis estimation model M1 outputs "High" as the prognosis estimate D2 if the probability of the patient surviving for a predetermined period is above a predetermined threshold, and "Low" if it is below the predetermined threshold. The prognosis estimate D2 may also indicate the period during which the probability of survival for the patient being estimated from the time the sample was obtained is estimated to be above a predetermined threshold.

[0023] The information processing device 1 can use the spatial distribution of biomarkers or predetermined proteins in a sample to predict a patient's prognosis, thereby improving the accuracy of disease prognosis prediction compared to existing prediction techniques.

[0024] [Configuration of Information Processing Device 1] Figure 2 is a block diagram showing the configuration of the information processing device 1. The information processing device 1 includes a communication unit 11, a storage unit 12, and a control unit 13. The control unit 13 includes an acquisition unit 131, a prognosis estimation unit 132, a distribution generation unit 133, a registration unit 134, and a learning unit 135.

[0025] The communication unit 11 is a communication interface for sending and receiving data with other devices via a network. The storage unit 12 is a storage medium including ROM (Read Only Memory), RAM (Random Access Memory), SSD (Solid State Drive), hard disk drive, etc. The storage unit 12 pre-stores programs to be executed by the control unit 13.

[0026] The memory unit 12 stores a prognosis estimation model M1 that has been trained to output the prognosis of a patient when it receives input data that includes the spatial distribution of a predetermined biomarker and at least one of a predetermined protein in a sample taken from the patient. The prognosis estimation model M1 is a trained model that has been trained using the spatial distribution of biomarkers or proteins in a sample taken from the patient and the patient's prognosis as training data. In the prognosis estimation model M1, the predetermined biomarker features and predetermined proteins corresponding to the spatial distribution acquired by the acquisition unit 131 have been trained as training data.

[0027] The control unit 13 is a processor, such as a CPU (Central Processing Unit). By executing a program stored in the memory unit 12, the control unit 13 functions as an acquisition unit 131, a prognosis estimation unit 132, a distribution generation unit 133, a registration unit 134, and a learning unit 135.

[0028] The acquisition unit 131 acquires the spatial distribution of at least one of the feature quantities related to a predetermined biomarker and a predetermined protein in a sample collected from the target patient. The acquisition unit 131 may acquire the spatial distribution of either the feature quantities related to the predetermined biomarker or the predetermined protein, or it may acquire the spatial distribution of both. The acquisition unit 131 may acquire the spatial distribution of the feature quantities related to the predetermined biomarker, etc., from an external device (not shown). The acquisition unit 131 may acquire the spatial distribution generated by image analysis of the acquired sample image data, as described later.

[0029] The predetermined biomarkers are, for example, high-frequency microsatellite instability or BRAF gene mutations, but are not limited to these. The predetermined biomarkers may also be low-frequency microsatellite instability, KRAS, SYNE1 (Spectrin Repeat Containing Nuclear Envelope Protein 1), APC (antigen-presenting cells), TP53, or TTN, etc. The acquisition unit 131 may acquire the spatial distribution of each of the multiple types of biomarkers. The features related to the predetermined biomarker may be the distribution of the biomarker itself, or it may be the distribution of information (e.g., Attention Weight) that indicates the contribution to the estimation result of the degree of biomarker expression in a machine learning model that estimates the degree of biomarker expression from image data of the input sample to be judged.

[0030] The specified protein is, for example, a CD3-positive lymphocyte or a CD20-positive lymphocyte, but is not limited to these. The specified protein may also be a CD4-positive lymphocyte, a CD8-positive lymphocyte, Foxp3, PD-1, CD163Ave, CD155, etc. The acquisition unit 131 may acquire the spatial distribution of each of multiple types of proteins.

[0031] The prognosis estimation unit 132 inputs the input data, including the spatial distribution acquired by the acquisition unit 131, into the prognosis estimation model M1, and outputs the information as an estimated value of the patient's prognosis. With the information processing device 1 configured in this way, the spatial distribution of biomarkers or predetermined proteins in the sample can be used for prediction, which has the effect of improving the accuracy of disease prognosis prediction compared to existing prediction techniques.

[0032] [Estimation of biomarker distribution] The information processing device 1 may be configured to generate a spatial distribution of features related to a predetermined biomarker in a sample based on image data of the sample. Figure 3 shows an example of the processing performed by the distribution generation unit 133 to estimate the spatial distribution of features related to the biomarker. First, the learning process for estimating the spatial distribution of features related to the biomarker will be described. In the learning process, the acquisition unit 131 acquires image data P11 of the sample and the ground truth label L assigned to the image data as training data. In the learning process, MIL (Multiple Instance Learning) may be used as an example. The ground truth label L is quantitative or qualitative information about the biomarker in the entire sample being imaged. As an example, the ground truth label L is information indicating the degree of microsatellite instability in the entire sample.

[0033] The acquisition unit 131 divides the acquired sample image data P11 into tiles. The learning unit 135 inputs the multiple image data P12 obtained by dividing the image data P11 into a distribution estimation model M2 and outputs a classification result R1. The classification result R1 is information corresponding to the correct label L, and is a value estimated by the distribution estimation model M2 based on the multiple image data P12. The learning unit 135 feeds back the difference between the output classification result R1 and the correct label L to the distribution estimation model M2 and updates the parameters of the distribution estimation model M2. The learning unit 135 repeats the above process until the conditions for the end of learning are met, and stores the trained distribution estimation model M2 in the storage unit 12. As a result, when the image data of a sample is input to the storage unit 12, it stores the distribution estimation model M2 that has been trained to output the spatial distribution of features related to a predetermined biomarker in the image data.

[0034] Next, the inference process will be explained. The acquisition unit 131 acquires image data P13 of a sample taken from the patient to be estimated. The distribution generation unit 133 inputs the image data of the sample acquired by the acquisition unit 131 into the distribution estimation model M2 and generates a spatial distribution. Specifically, the distribution generation unit 133 divides the acquired image data P13 into tiles and inputs them into the distribution estimation model M2. The distribution generation unit 133 acquires the Attention Weight(A) when the distribution estimation model M2 estimates the classification result R2 of the image data P13. The Attention Weight(A) is a value that indicates the degree to which each part of the image data contributed to the classification when classifying the image data. The Attention Weight(A) values ​​generated in this way indicate the degree of contribution to inference corresponding to the position in the image space, and can therefore be used as a spatial distribution of features related to biomarkers.

[0035] With the information processing device 1 configured in this way, it is possible to generate a spatial distribution of features related to a predetermined biomarker in a sample, enabling highly accurate prognosis prediction using the spatial distribution of features related to the predetermined biomarker, compared to existing prediction techniques.

[0036] [Generating the spatial distribution of proteins] Next, the process for generating the spatial distribution of a predetermined protein will be explained using Figure 4. The acquisition unit 131 acquires first sample image data P21 and second sample image data P22 from a sample taken from the patient to be estimated. The first sample image data P21 is image data obtained by processing (e.g., staining) a sample taken from the patient to be estimated in a predetermined way so that the composition of cells or tissues can be detected, and then imaging the sample. The second sample image data P22 is image data obtained by processing a sample taken from the patient to be estimated in a predetermined way so that a predetermined protein in the sample can be detected, and then imaging the sample. Image data P21 and image data P22 are, as an example, image data generated by slicing a collected sample so that the cross-sections are parallel and the thickness is constant, staining the sliced ​​sample in a predetermined way, and then imaging the cross-section of the sample. The first sample image data P21 and the second sample image data P22 may be stained and imaged using different methods. The method of staining the sample is, for example, HE staining for the first sample image data P21 and IHC staining for the second sample image data P22, but is not limited to this. In the image data for the first sample (P21) and the image data for the second sample (P22), images were captured of the cross-sections that were adjacent to each other before the samples were sliced.

[0037] The registration unit 134 associates corresponding positions in image data. The registration unit 134 associates the position in the first sample image data P21 acquired by the acquisition unit 131 with the position in the second sample image data P22. As an example, the registration unit 134 associates image data by using a known non-rigid registration to transform one of the image data so that a pixel in one image data matches a corresponding pixel in the other image data.

[0038] The memory unit 12 stores the tumor region extraction model M31 and the protein extraction model M32. The tumor region extraction model M31 is a trained model that, upon input of the first sample image data P21, outputs the tumor regions occurring in the sample imaged in the said image data. The learning unit 135 has previously trained the tumor region extraction model M31 using the first sample image data and tumor regions as training data.

[0039] The protein extraction model M32 is a pre-trained model that, upon input of the second sample image data P22, outputs the region where a predetermined protein expressed in the image data is expressed in the sample captured in the image data. The learning unit 135 has previously trained the protein extraction model M32 using the second sample image data and the region where the predetermined protein is expressed in the image data as training data.

[0040] The distribution generation unit 133 generates a spatial distribution of tumor tissue and a predetermined protein in a sample based on the first sample image data and the second sample image data acquired by the acquisition unit 131. Specifically, the distribution generation unit 133 inputs the first sample image data P21 and the second sample image data P22 acquired by the acquisition unit 131 into the tumor region extraction model M31 and the protein extraction model M32, respectively, to output the tumor region D11 and the region D12 where the predetermined protein is expressed. The tumor region D11 and the region D12 where the predetermined protein is expressed, output by the tumor region extraction model M31 and the protein extraction model M32, respectively, correspond to their positions in the image data. Therefore, the tumor region D11 and the region D12 where the predetermined protein is expressed represent the spatial distribution of tumor tissue and the predetermined protein in the image data, respectively. The distribution generation unit 133 outputs the spatial distribution of tumor tissue and the predetermined protein to the acquisition unit 131.

[0041] The memory unit 12 stores a prognosis estimation model M1 that has been learned using the spatial distribution of tumor tissue and predetermined proteins in a sample taken from a patient as further input. The acquisition unit 131 further acquires the spatial distribution of tumor tissue and predetermined proteins in a sample taken from the target patient. The acquisition unit 131 may also acquire the spatial distribution of tumor tissue and predetermined proteins generated by the distribution generation unit 133 based on first sample image data and second sample image data obtained from the target patient's sample.

[0042] The prognosis estimation unit 132 inputs the input data, which further includes the spatial distribution of tumor tissue acquired by the acquisition unit 131, into the prognosis estimation model M1, and outputs the information as an estimated prognosis for the patient. With the information processing device 1 configured in this way, it becomes possible to predict the prognosis using information that associates the distribution of tumor tissue with the distribution of a predetermined protein, thereby enabling highly accurate predictions.

[0043] [Processing flow in information processing device 1] Figure 5 is a flowchart illustrating an example of the processing flow in the information processing device 1. The flowchart shown in Figure 5 begins from the point when an instruction to start estimation processing is received from an external device.

[0044] The acquisition unit 131 acquires image data of multiple samples (S01). The registration unit 134 registers and associates each acquired image data (S02).

[0045] The distribution generation unit 133 generates a spatial distribution of features related to a predetermined biomarker based on the acquired image data (S03). The distribution generation unit 133 generates a spatial distribution of tumor regions based on the acquired image data (S04). The distribution generation unit 133 generates a spatial distribution of a predetermined protein based on the acquired image data (S05).

[0046] The prognosis estimation unit 132 inputs each spatial distribution generated by the distribution generation unit 133 into the prognosis estimation model M1 (S06). The prognosis estimation unit 132 outputs the estimated value output by the prognosis estimation model M1 as the prognosis estimate (S07). The information processing device 1 then terminates processing.

[0047] <Example 1> In the above explanation, an example of predicting the prognosis of a patient with a specific disease was described. However, the information processing device 1 may also be configured as a device for estimating whether or not a specific drug will be effective for a patient with a specific disease. In the following, the same reference numerals as those already described will be used, and their explanations will be omitted.

[0048] Figure 6 shows an overview of the processing of the information processing device 1 according to Modification 1. The information processing device 1 in the Modification differs from the information processing device 1 shown in Figure 1 in that it further acquires drug information D3 and inputs the acquired drug information into the prognosis estimation model M11 to obtain the prognosis estimate value D2.

[0049] Specifically, the acquisition unit 131 further acquires the drugs to be administered to the target patient. For example, the acquisition unit 131 acquires drug information D3 indicating the drugs to be administered to the patient from an external device (not shown). Drug information D3 may be information indicating one type of drug, or it may be information indicating multiple types of drugs. Furthermore, drug information D3 may include information indicating the type of drug and the usage, dosage, etc., of the drug to be administered.

[0050] The memory unit 12 may store a prognosis estimation model M11 that has been further trained using drugs administered to the patient as input. In this case, the prognosis estimation model M11 stored by the memory unit 12 is a trained model that has been trained using the spatial distribution of features related to a predetermined biomarker in a patient sample used for training, drug information indicating the drugs administered to the patient, and information indicating the patient's prognosis as training data. The prognosis estimation model M11 stored by the memory unit 12 takes the spatial distribution of features related to a predetermined biomarker in a sample taken from the patient to be judged, and drug information D3 indicating the drugs administered to the patient as input, and outputs a prognosis estimate D2 indicating the patient's prognosis.

[0051] The prognosis estimation unit 132 further inputs the drug information D3, which indicates the drugs to be administered to the target patient, acquired by the acquisition unit 131, into the prognosis estimation model M11, and outputs the output information as the prognosis estimate value D2 for the target patient. In addition to the spatial distribution, the prognosis estimation unit 132 inputs the information indicating the drugs to be administered to the patient, acquired by the acquisition unit 131, into the prognosis estimation model M11 stored in the storage unit 12, and outputs the prognosis estimate value D2 output from the prognosis estimation model M11.

[0052] [Effects of the information processing device 1 according to modified example 1] With the information processing device 1 configured in this way, the accuracy of estimating whether or not a drug is effective for a patient with a predetermined disease can be improved.

[0053] <Modification 2> It is known that different drugs are effective depending on the heterogeneity of the tumor tissue (the presence of various types of tissue). Therefore, by configuring the information processing device 1 to take into account the distribution of features related to the tumor tissue and predict the prognosis of the target patient, it is possible to make estimations that take into account the differences in prognosis due to the heterogeneity of the tumor tissue.

[0054] Figure 7 shows an example of processing in the information processing device 1 according to a modified example. In this case, the storage unit 12 stores a prognosis estimation model M12 that has been learned by further inputting the spatial distribution of features related to tumor tissue in a sample taken from a patient. The distribution generation unit 133 generates tumor region image data P32 based on the first sample image data P31 and the spatial distribution of the tumor region in the first sample image data P31. The tumor region image data P32 is image data that includes information only on the region where the tumor is present from the first sample image data P31.

[0055] The distribution generation unit 133 inputs tumor region image data P32 to the tumor region classification model M41 and outputs feature quantities D21 for each region (hereinafter referred to as "patch") obtained by micronizing the tumor region in the tumor region image data P32. Feature quantities D21 are, for example, cell density in tumor cells or labels indicating similar images on a patch-by-patch basis. The tumor region classification model M41 is a pre-trained model that the learning unit 135 has trained to output feature quantities for each patch using tumor region image data as training data.

[0056] The acquisition unit 131 acquires the spatial distribution of feature quantities related to tumor tissue in the sample taken from the target patient. For example, the acquisition unit 131 acquires the feature quantities D21 for each patch obtained by micro-dividing the tumor region generated by the distribution generation unit 133 as the spatial distribution of feature quantities related to tumor tissue in the sample taken from the target patient. The acquisition unit 131 may also acquire the spatial distribution of feature quantities related to tumor tissue in the sample taken from the target patient from an external device.

[0057] The prognosis estimation unit 132 outputs the information obtained by inputting the input data, including the spatial distribution of features related to tumor tissue acquired by the acquisition unit 131, into the prognosis estimation model M12, and outputs the estimated prognosis of the target patient. In addition to the spatial distribution of features related to tumor tissue, the prognosis estimation unit 132 may further input the spatial distribution of features related to predetermined biomarkers, etc., into the prognosis estimation model M12 and output the estimated prognosis D2.

[0058] With the information processing device 1 configured, it becomes possible to make estimations that take into account differences in prognosis due to heterogeneity of tumor tissue.

[0059] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of its gist. For example, all or part of the apparatus can be configured by functionally or physically distributing and integrating in any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments resulting from the combinations are combined with the effects of the original embodiments. [Explanation of Symbols]

[0060] 1. Information Processing Device 11 Communications Department 12 Storage section 13 Control Unit 131 Acquisition Department 132 Prognostic Estimation Section 133 Distribution generator 134 Registration Department 135 Learning Department

Claims

1. A prognostic estimation model, trained to output the patient's prognosis when input data is provided that includes the spatial distribution of features related to a predetermined biomarker in a sample taken from the patient, the spatial distribution of tumor tissue, and the spatial distribution of a predetermined protein, and the prognosis of the patient. When a first sample image data, which is image data of a sample taken from a target patient and which has been processed in a predetermined way to detect the composition of cells or tissues in the sample, is input, a tumor region extraction model that has been trained to output tumor regions occurring in the sample as captured in the first sample image data is used, When a second sample image data, which is image data of a sample taken from the aforementioned patient and processed in a predetermined manner to detect a predetermined protein, is input, a protein extraction model trained to output the region in the sample where the predetermined protein is expressed as captured in the second sample image data is provided, When a third sample image data, which is image data of a sample taken from the aforementioned patient, is input, a distribution estimation model trained to output the spatial distribution of features related to the predetermined biomarker in the third sample image data is used. A memory unit that stores, An acquisition unit that acquires the first sample image data, the second sample image data, and the third sample image data, The acquisition unit inputs the first sample image data and the second sample image data acquired by the acquisition unit into the tumor region extraction model and the protein extraction model, respectively, and generates the spatial distribution of tumor tissue and predetermined proteins in the sample based on the output results; the acquisition unit inputs the third sample image data acquired by the acquisition unit into the distribution estimation model and generates the spatial distribution of features related to the predetermined biomarker into the distribution generation unit; A prognosis estimation unit that outputs an estimated prognosis for the aforementioned patient, It has, The acquisition unit acquires the spatial distribution of at least one of the spatial distributions of feature quantities relating to a predetermined biomarker, the spatial distribution of the tumor tissue, and the spatial distribution of a predetermined protein in the sample collected from the target patient, which is generated by the distribution generation unit. The prognosis estimation unit outputs the information obtained by inputting the input data, including the spatial distribution acquired by the acquisition unit, into the prognosis estimation model as an estimated value of the prognosis for the target patient. Information processing device.

2. The memory unit stores the prognosis estimation model, which has been further trained using the drugs administered to the patient as input. The acquisition unit further acquires the drug to be administered to the target patient, The prognosis estimation unit further inputs the drugs to be administered to the target patient, acquired by the acquisition unit, into the prognosis estimation model, and outputs the output information as an estimated value of the target patient's prognosis. The information processing apparatus according to claim 1.

3. The spatial distribution of the feature quantities relating to the predetermined biomarker is the spatial distribution of the feature quantities relating to high-frequency microsatellite instability or BRAF gene mutation in the sample collected from the patient. The information processing apparatus according to claim 1 or 2.

4. The spatial distribution of the predetermined protein is the spatial distribution of tumor tissue and CD3-positive lymphocytes or CD20-positive lymphocytes in the sample collected from the patient. The information processing apparatus according to claim 1 or 2.

5. The memory unit stores the prognosis estimation model, which has been learned by further inputting the spatial distribution of tumor tissue and predetermined proteins in a sample taken from the patient. The acquisition unit further acquires the spatial distribution of tumor tissue and a predetermined protein in a sample taken from the target patient. The prognosis estimation unit outputs the information obtained by inputting the input data, which further includes the spatial distribution of the tumor tissue and predetermined proteins acquired by the acquisition unit, into the prognosis estimation model, and outputs the resulting information as an estimated prognosis for the patient in question. The information processing apparatus according to claim 1 or 2.

6. The first sample image data and the second sample image data are image data obtained by staining samples collected from the patient using different methods, respectively. The aforementioned information processing device is The system further includes a registration unit that associates the position in the first sample image data with the position in the second sample image data. The distribution generation unit generates a spatial distribution of tumor tissue and a predetermined protein in the sample based on the first sample image data and the second sample image data associated with the registration unit. The information processing apparatus according to claim 5.

7. The memory unit stores the prognosis estimation model, which has been learned by further inputting the spatial distribution of features related to tumor tissue in a sample taken from a patient. The acquisition unit further acquires the spatial distribution of characteristic quantities related to tumor tissue in the sample taken from the target patient, The prognosis estimation unit outputs the information obtained by inputting the input data, which further includes the spatial distribution of the tumor tissue features acquired by the acquisition unit, into the prognosis estimation model, and outputs the resulting information as an estimated prognosis for the patient in question. The information processing apparatus according to claim 1 or 2.

8. The aforementioned prognosis estimation model is trained on training data that correlates the spatial distribution of biomarkers or proteins in a sample taken from a patient with the patient's prognosis, which indicates whether or not the patient survived for a predetermined period from the time the sample was obtained. The information processing apparatus according to claim 1.

9. The memory unit, upon inputting image data of a sample taken from a patient, further stores a distribution estimation model that has been trained to output the spatial distribution of features relating to a predetermined biomarker in the image data, and which is based on training data that associates image data of a training sample with ground truth data indicating the degree of the predetermined biomarker in the entire sample shown by the image data. The acquisition unit further acquires image data of the sample collected from the target patient, The information processing device further includes a distribution generation unit that inputs the image data into the distribution estimation model and generates the spatial distribution. The information processing apparatus according to claim 1.

10. The tumor region extraction model is trained on training data which associates training sample image data with information indicating tumor regions in the sample image data, The protein extraction model is trained based on training data that associates training sample image data with information indicating regions in the image data where a predetermined protein is expressed. The information processing apparatus according to claim 1.

11. A computer executes Steps include referring to a storage unit that stores: a prognosis estimation model trained to output the patient's prognosis when input data including the spatial distribution of features related to a predetermined biomarker in a sample taken from a patient, the spatial distribution of tumor tissue, and the spatial distribution of a predetermined protein; a tumor region extraction model trained to output tumor regions occurring in the sample imaged in the first sample image data when input first sample image data, which is image data of a sample taken from a target patient that has been processed in a predetermined way to detect the composition of cells or tissues in the sample; a protein extraction model trained to output regions where a predetermined protein is expressed in the sample imaged in the second sample image data when input second sample image data, which is image data of a sample taken from the target patient that has been processed in a predetermined way to detect a predetermined protein in the sample; and a distribution estimation model trained to output the spatial distribution of features related to the predetermined biomarker in the third sample image data when input third sample image data, which is image data of a sample taken from the target patient. The steps include obtaining the first sample image data, the second sample image data, and the third sample image data, The steps include: inputting the first sample image data and the second sample image data acquired in the acquisition step into the tumor region extraction model and the protein extraction model, respectively, and generating the spatial distribution of tumor tissue and predetermined proteins in the sample based on the output results; inputting the third sample image data acquired in the acquisition step into the distribution estimation model and generating the spatial distribution of features related to the predetermined biomarker; Steps to obtain the spatial distribution of features related to a predetermined biomarker, the spatial distribution of the tumor tissue and the spatial distribution of a predetermined protein, and at least one of the spatial distributions in the sample taken from the target patient generated in the step of generating the spatial distribution, The step of inputting the input data, including the spatial distribution obtained in the step of obtaining the spatial distribution, into the prognosis estimation model, and outputting the information output as an estimated value of the prognosis of the target patient, An information processing method having

12. On the computer, Steps include referring to a storage unit that stores: a prognosis estimation model trained to output the patient's prognosis when input data including the spatial distribution of features related to a predetermined biomarker in a sample taken from a patient, the spatial distribution of tumor tissue, and the spatial distribution of a predetermined protein; a tumor region extraction model trained to output tumor regions occurring in the sample imaged in the first sample image data when input first sample image data, which is image data of a sample taken from a target patient that has been processed in a predetermined way to detect the composition of cells or tissues in the sample; a protein extraction model trained to output regions where a predetermined protein is expressed in the sample imaged in the second sample image data when input second sample image data, which is image data of a sample taken from the target patient that has been processed in a predetermined way to detect a predetermined protein in the sample; and a distribution estimation model trained to output the spatial distribution of features related to the predetermined biomarker in the third sample image data when input third sample image data, which is image data of a sample taken from the target patient. The steps include obtaining the first sample image data, the second sample image data, and the third sample image data, The steps include: inputting the first sample image data and the second sample image data acquired in the acquisition step into the tumor region extraction model and the protein extraction model, respectively, and generating the spatial distribution of tumor tissue and predetermined proteins in the sample based on the output results; inputting the third sample image data acquired in the acquisition step into the distribution estimation model and generating the spatial distribution of features related to the predetermined biomarker; Steps to obtain the spatial distribution of features related to a predetermined biomarker, the spatial distribution of the tumor tissue and the spatial distribution of a predetermined protein, and at least one of the spatial distributions in the sample taken from the target patient generated in the step of generating the spatial distribution, The step of inputting the input data, including the spatial distribution obtained in the step of obtaining the spatial distribution, into the prognosis estimation model, and outputting the information output as an estimated value of the prognosis of the target patient, A program that executes the command.

Images