Computer system, model learning method, and information processing method
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- HITACHI LTD
- Filing Date
- 2023-12-27
- Publication Date
- 2026-08-06
AI Technical Summary
Clinical data typically has a low number of samples for input parameters, leading to insufficient prediction accuracy and potential overfitting in machine learning models, and existing transfer learning methods do not adequately address this issue.
A computer system employs transfer learning using a first base model generated from a large-scale public database, incorporating a third loss function to adjust the influence of learning data based on distribution characteristics and a fifth loss function to evaluate output differences, enhancing the prediction model's accuracy.
The method improves prediction accuracy by effectively utilizing a base model, even with limited clinical data, and incorporates clinically influential features, resulting in higher precision.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to transfer learning for improving the accuracy of a prediction model that handles data with a small amount of information (e.g., actual clinical data) by utilizing a prediction model generated using a large-scale public database.
Background Art
[0002] With the progress of gene measurement technologies represented by next-generation sequencing (NGS), high-throughput screening has become possible, and in recent years, large-scale registries such as gene sequence data have been rapidly accumulating. For example, many clinical trial data including master clinical trials sponsored by medical institutions such as TCGA (The Cancer Genome Atlas, NCI: National Cancer Institute), public cell line data (CCLE: Cancer Cell Line Encyclopedia, GDSC: Genomics of Drug Sensitivity in Cancer, etc.), patient tumor xenograft (PDX) data, and the like.
[0003] Analytical research using these so-called medical big data is also very active. For example, research on a drug sensitivity prediction model using public cell line data is one of them. However, the accuracy of current prediction models and, in addition, the method of interpreting the results obtained therefrom do not have sufficient clinical predictive power. This is presumably because commonly used cell lines and xenografts are models that cannot effectively mimic / predict human reactions (see Non-Patent Document 1).
[0004] On the other hand, cell line data is a huge amount of actual case data of approximately 1000 in number, approximately 300 drugs, and approximately 240,000 drug efficacy experiments (see Non-Patent Document 2), and useful utilization methods are desired.
[0005] Regarding the issues of verification accuracy in publicly available cell line data, drug response profiling using patient-derived xenografts (PDX) is progressing, but there is a problem that the creation of PDX is costly and time-consuming.
[0006] In Non-Patent Document 2, a prediction model learned from publicly available cell line data was transferred and learned to in-vivo tumor and ex-vivo patient-derived xenograft (PDX) data to construct a model for predicting the outcome. It is reported that in the prediction for each drug and the prediction for each cancer, the AUROC was improved by 49.8% and 47.8% respectively compared to the case without using transfer learning (average).
Prior Art Documents
Non-Patent Documents
[0007]
Non-Patent Document 1
Non-Patent Document 2
Non-Patent Document 3
[0008] For example, in clinical practice, medical data generally has an extremely low number of samples (N) for the input parameters (explanatory variables, features) (P) (P >> N). Therefore, a prediction model using machine learning may not achieve sufficient prediction accuracy or is highly likely to cause overfitting. Thus, for example, the clinical application of a high-precision prediction model utilizing a large-scale database typified by publicly available cell line data is desired.
[0009] As the transfer learning method used in Non-Patent Document 2, a general transfer learning method of fine-tuning the final layer is utilized. Also, in the publicly available cell line data, a prediction model is generated without distinguishing by the target disease and using all cell line data as teacher data.
[0010] An object of the present invention is to realize a system and method for generating a prediction model with high prediction by transfer learning.
Means for Solving the Problems
[0011] In the present application, a typical example of the disclosed invention is as follows. That is, a computer system including a computer having a processor, a storage device connected to the processor, and a connection interface connected to the processor, wherein a first base model is generated by machine learning using a first learning dataset stored in the storage device, and a second model is generated by transfer learning using the first model and a second learning dataset stored in the storage device. In the transfer learning, a third loss function including a second loss function for adjusting the influence of the learning data on the learning according to the distribution characteristics of the items corresponding to the prediction target of the second model included in the learning data constituting the second learning dataset, and a fifth loss function including a fourth loss function for evaluating the difference between the outputs of the first model and the second model are used, and at least one of them is used to generate the second model.
Effects of the Invention
[0012] According to the present invention, effective transfer learning using a base model (first model) can be realized. As a result, the prediction accuracy of the prediction model can be improved. Problems, configurations, and effects other than those described above will be clarified by the description of the following embodiments.
Brief Description of Drawings
[0013]
Figure 1
Figure 2
Figure 3
Figure 4A
Figure 4B
Figure 5A
Figure 5B
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14
Figure 15
Mode for Carrying Out the Invention
[0014] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not to be construed as being limited to the description of the embodiments shown below. It will be readily understood by those skilled in the art that the specific configuration can be changed without departing from the spirit or gist of the present invention.
[0015] In the configuration of the invention described below, the same or similar configurations or functions are denoted by the same reference numerals, and duplicate descriptions are omitted.
[0016] Expressions such as "first", "second", "third", etc. in this specification and the like are attached to identify components, and do not necessarily limit the number or order.
[0017] In the drawings and the like, the positions, sizes, shapes, and ranges of the respective configurations shown may not represent the actual positions, sizes, shapes, and ranges, etc. in order to facilitate understanding of the invention. Therefore, in the present invention, it is not limited to the positions, sizes, shapes, and ranges, etc. disclosed in the drawings and the like.
Embodiment
[0018] FIG. 1 is a diagram showing an example of the configuration of the computer system in Example 1.
[0019] The computer system is composed of a computer 100, an information terminal 110, and an external storage device 111. The computer 100, the information terminal 110, and the external storage device 111 are connected to each other via a network 109. The network 109 is, for example, a LAN (Local Area Network) and a WAN (Wide Area Network), etc., and the connection method may be either wired or wireless.
[0020] The computer 100 executes machine learning to generate a prediction model for making predictions regarding a patient's disease / condition (such as diagnostic prediction, optimal treatment prediction, prognosis prediction, etc.), and presents prediction results for actual clinical data (input data 220) using the model. The computer 100 includes a CPU 101, a main memory device 102, a secondary storage device 103, a network adapter 104, an input device 105, and an output device 106. Each hardware element is connected to each other via an internal bus 108.
[0021] The CPU 101 executes a program stored in the main memory device 102. By executing processing according to the program, the CPU 101 operates as a functional unit (module) that realizes a specific function. In the following description, when explaining processing with a functional unit as the subject, it indicates that the CPU 101 is executing a program that realizes the functional unit.
[0022] The main memory device 102 is a DRAM (Dynamic Random Access Memory) and stores the program executed by the CPU 101 and the data used by the program. The main memory device 102 is also used as a work area.
[0023] The secondary storage device 103 is an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc., and permanently stores data. The program and data stored in the main memory device 102 may be stored in the secondary storage device 103. In this case, the CPU 101 reads the program and information from the secondary storage device 103 and loads them into the main memory device 102.
[0024] The network adapter 104 is an interface for connecting to an external device via the network 109.
[0025] The input device 105 is a keyboard, a mouse, a touch panel, etc., and is a device for inputting to the computer 100.
[0026] The output device 106 is a device such as a display and a printer, and is a device for outputting the processing result of the computer 100 and the like.
[0027] Note that the hardware configuration of the computer 100 is an example and is not limited to this. For example, the computer 100 may not have the input device 105 and the output device 106.
[0028] The information terminal 110 is a terminal for performing various operations on the computer 100. For example, the information terminal 110 performs registration / deletion of learning data, registration of a model, input of user data, and the like. The hardware configuration of the information terminal 110 is the same as that of the computer 100.
[0029] The external storage device 111 stores various information. The external storage device 111 is, for example, an external HDD or a storage system.
[0030] FIG. 2 is a diagram showing an example of the software configuration of the computer 100 according to the first embodiment.
[0031] The computer 100 includes a first learning unit 200, a second learning unit 201, and a prediction unit 202, and also includes a first learning data DB 210, a second learning data DB 211, and a model DB 212. Note that the first learning data DB 210, the second learning data DB 211, and the model DB 212 may be stored in the external storage device 111.
[0032] The first learning data DB 210 is a database that stores learning data used for machine learning of the base model. The first learning data DB 210 is a large-scale public database that stores public clinical data, clinical trial data, cell line data, and the like.
[0033] The second learning data DB 211 is a database that stores learning data used for transfer learning. The learning data is data obtained in actual clinical practice, and includes, for example, items such as a patient's age, gender, blood pressure, test results, genetic information, treatment results, and prognosis information.
[0034] The model DB212 is a database that stores model information. The model information is, for example, hyperparameters such as the structure and weights of the model.
[0035] The first learning unit 200 performs machine learning using the learning data stored in the first learning data DB210 and the base model stored in the model DB212, and stores the learned base model in the model DB212.
[0036] The second learning unit 201 performs transfer learning using the second learning data DB211 and the base model stored in the model DB212, and stores the generated prediction model in the model DB212.
[0037] The prediction unit 202 receives the input data 220 using the prediction model stored in the model DB212, makes a prediction such as the medical effect of the patient, and outputs it as the prediction result 221.
[0038] Next, the learning method of the prediction model of Example 1 will be described.
[0039] Figure 3 illustrates an example of the input and output of the base model and the prediction model of Example 1.
[0040] The input / output 301 shows the input / output of a base model (drug efficacy prediction model) learned using publicly available cell line data. The input / output 302 shows the input / output of a prediction model (drug efficacy prediction model) obtained by transfer learning of the base model using clinical trial data (Beat AML data publicly available from the Leukemia and Lymphoma Society).
[0041] The input data includes gene mutations, expression levels, gene copy number mutations, and drug targets, and the output data is the IC50 value (the concentration of the drug required to halve the cells) for the drug target. The input and output are not limited to this, and appropriate input and output models can be set according to the problem setting. For example, it is conceivable to use gene structural polymorphisms, drug-related genes, the chemical structure of drugs, and treatment history information as input, and RECIST, test values, or prognosis information as output.
[0042] Transfer learning is generally a method of solving a problem in the target domain by using a solution method (pre-trained model) in a different but related source domain (see, for example, Non-Patent Document 1). In the case of the input and output in Figure 3, the task of solving using the Beat AML data is the target domain, and the task of solving using the public cell line database is the source domain.
[0043] Figures 4A and 4B are diagrams showing an example of the structures of the base model and the prediction model of Example 1.
[0044] Network 401 is an example of a base model that is the learning target in the source domain. Network 402 is an example of a prediction model that is the learning target in the target domain.
[0045] Note that there is no guarantee that the data handled in the source domain and the target domain are of the same type and characteristics. In this example, gene copy number variation data (CNV) existing in the public cell line data is used for the learning of the base model. This is incorporated as a feature of the base model based on the medical knowledge that CNV has a high influence on drug efficacy prediction. CNV related to the actual target tumor is not input to the prediction model.
[0046] The computer 100 learns a base model (network 401) using learning data (public cell line data) stored in the first learning data DB 210. The computer 100 sets the parameters of each layer of the base model as the initial values of the parameters of each layer of a prediction model (network 402), and learns the prediction model (network 402) using learning data (clinical trial data) stored in the second learning data DB 211.
[0047] Note that a fixed value may be set as the initial value of the parameters of at least one layer. Also, a fixed value may be set as the initial values of the parameters of all layers. When setting the initial values of the parameters of all layers to fixed values, a method of learning only the last layer (identification layer) 404 may be used. Also, when using a domain adaptation method (see, for example, Non-Patent Document 4), only the initial value of the first input layer 403 of the network 402 may be randomly set.
[0048] FIG. 5A is a diagram for explaining the distribution of the objective variable of the clinical trial data (Beat AML data) of Example 1. FIG. 5B is a diagram for explaining the distribution of the objective variable in the prediction of the prediction model of Example 1. FIG. 6 is a diagram showing an example of the clinical trial data (Beat AML data) of Example 1.
[0049] In the case database in actual clinical practice, there are many cases that are not standardized, such as disease names, drug names, and the expression methods of gene mutations. For example, clinical trial data such as Beat AML is obtained in accordance with actual trial conditions, so there are cases where it is difficult for the side that utilizes the database later to understand the intention.
[0050] Fig. 6 shows an example of Beat AML data. In this efficacy test, each drug is administered to the specimens in vitro, and the amount of drug required to inhibit in each of the cases of 10%, 25%, 50%, 75%, and 90% is examined. However, the amount of drug to be examined is limited within a certain range. This is presumably due to the fact that in actual cases, the amount of specimens that can be used is small compared to cell lines that can be cultured. In Fig. 6, the minimum value (min_conc) and the maximum value (max_conc) of the amount of drug for which the efficacy test was conducted are set, but at the time of IC50, it exceeds the maximum value (max_conc) that can be experimentally achieved. That is, there is data where the experiment was terminated at IC50 = max_conc. Among the data used in this example, out of 17,000 cases, there were about 7,000 cases of the above-mentioned data.
[0051] The above-mentioned data is considered not to be useful for the learning of the prediction model. Therefore, in this example, a loss function (second loss function) is introduced to suppress the influence (parameter update) of the above-mentioned data on learning. For example, the loss function shown in Equation (1) can be considered. By this, the influence on the learning of the learning data can be adjusted according to the distribution characteristics of the target variable included in the learning data.
[0052]
Equation
[0053] The graph in Fig. 5A represents the distribution of cases of -log 10 IC50. In this graph, α is the value corresponding to the maximum value of IC50, and β corresponds to the minimum value of IC50. The loss function L shown in Equation (1) acts so that the loss of the predicted value regarding the learning data not within the range 501 defined by the values α and β does not increase. That is, learning is executed considering only the loss regarding the learning data whose values are included in the range 501.
[0054] Note that transfer learning may be performed using a loss function (third loss function) defined by combining the loss function (second loss function) shown in Expression (1) and the loss function (first loss function) for evaluating the prediction accuracy of the prediction model. The loss function for evaluating the prediction accuracy of the prediction model is, for example, cross entropy.
[0055] The graph of FIG. 5B shows the prediction distribution after learning using a loss function including the loss function of Expression (1). It can be seen that a natural data distribution is obtained even near the boundaries (α and β).
[0056] FIGS. 7 and 8 are diagrams showing the experimental results of the prediction model of Example 1.
[0057] FIG. 7 shows a comparison between a prediction model learned using only the Beat AML data and a prediction model generated by transfer learning using a publicly available cell line data learning model. Note that the transfer learning method used fine-tuning.
[0058] As in Non-Patent Document 1, the evaluation indices used were the RMSE shown in Expression (2) and the Spearman coefficient shown in Expression (3). Here, D represents the difference in ranks between the corresponding estimated value and the correct value, and N represents the number of value pairs.
[0059]
Number
[0060]
Number
[0061] As shown in FIG. 7, it can be seen that the prediction model of this example has higher accuracy than the conventional prediction model in both the RMSE and the Spearman coefficient.
[0062] Figure 8 shows a performance comparison between the method described in Non-Patent Document 1 and the method of Example 1. Although it is a comparison using the Spearman coefficient, numerical superiority can be observed. This is presumably due to the addition of the loss function according to the characteristics of the data distribution and the incorporation of CNV data features, as described in Example 1.
[0063] Figure 15 is a diagram showing an example of the GUI presented by the computer 100 of Example 1.
[0064] The GUI 1500 includes display columns 1501, 1502, 1503, 1504, and 1505.
[0065] The display column 1501 is a column for displaying information about the base model, and the version of the used base model, learning data, the number of data used for learning, and the performance index (AUC) are displayed.
[0066] The display column 1502 is a column for displaying information about the input data input to the prediction model, and the items, number, and information on the target drug included in the input data such as gene mutations and expression levels are shown.
[0067] The display column 1503 is a column for displaying information about the prediction model. In the display column 1504, the number of data finally used from the input data (the number of data after preprocessing by data cleaning due to missing / missing values, etc.), the method of transfer learning used, and the performance indices (AUC, Spensitivity / Specificity) are displayed.
[0068] The display column 1504 is a column for displaying information about the performance of the prediction model. The user can select various transfer learning methods, and the performance of the prediction model generated by each method is displayed in the display column 1504.
[0069] The display column 1506 displays the gene mutations corresponding to the drugs for which drug efficacy is expected. Also, as an interpretation thereof, for example, a pathway showing the relationship between the interaction of gene mutations and the drug is displayed.
[0070] According to Example 1, by performing transfer learning using a drug efficacy prediction model based on publicly available cell line data as a base model, the prediction accuracy for real clinical data can be improved. Furthermore, for the prediction target (output), data and its features that are highly clinically influential, which cannot be extracted by a prediction model using only real clinical data, a base model incorporating them is utilized, so an improvement in the prediction accuracy of the prediction model can be expected.
Example
[0071] In Example 2, the loss function used in transfer learning is different from that in Example 1. Hereinafter, Example 2 will be described centering on the differences from Example 1.
[0072] The configuration of the computer system in Example 2 is the same as that in Example 1. The hardware configuration and software configuration of computer 100 in Example 2 are the same as those in Example 1.
[0073] Figure 9 is a diagram for explaining the transfer learning in Example 2.
[0074] In Example 1, a prediction model was generated using a domain adaptation method. In Example 2, a method of knowledge distillation (see, for example, Non-Patent Document 5), which transfers the features (tacit knowledge) of learning, is proposed so that it can be applied to a clinical data utilization environment where the input data is further restricted.
[0075] In Figure 9, network 901 shows an example of a prediction model generated by the method of Example 1. Hereinafter, the prediction model generated by the method of Example 1 will be referred to as a teacher model.
[0076] Here, a CNN (Convolutional Neural Network) (VGG16 model in this example) is used as the prediction model. Note that the prediction model is not limited to this.
[0077] The input data 903 is data including, for example, gene mutations, expression levels, copy numbers, patient information, etc.
[0078] The network 902 is an example of a model that handles input data 904 with a restricted amount of information compared to the input data 903. The input data 904 is, for example, data that contains only gene mutations. Hereinafter, the prediction model corresponding to the network 902 will be described as a student model.
[0079] Here, the restriction on the amount of information of the input data means that the number of items included in the input data is less than the number of items included in the input data for comparison, or that the data amount of the values of the items included in the input data is smaller than the data amount of the values of the items included in the input data for comparison.
[0080] In the method of this embodiment, in addition to the output of the teacher model, knowledge regarding the feature amounts of the intermediate layers is also distilled into the teacher model. For distillation of the feature amounts of the intermediate layers, the FSP matrix (see Non-Patent Document 6), which is a known method, is used. By converting the outputs of a plurality of intermediate layers into a Gram matrix called the FSP Matrix, the FSP Matrix is generated for each of the teacher model and the student model, and by calculating the L2-Loss between the FSP Matrices, learning considering the relationship (flow of process) of the intermediate layers is realized.
[0081] In FIG. 9, the computer 100 calculates the Gram Matrices (G1 A , G3 A , G5 A , G1 B , G3 B , G5 B ) of each of the Block 1, 3, and 5 of the student model 902 of the teacher model 901 using Equation (4). In Equation (4), l represents the channel number, k represents the block number, and h and w are the width and depth of the convolutional layers C 1 , C 2 .
[0082]
Equation
[0083] The value of the Gram Matrix represents the correlation between the feature quantities output from each block. Also, the feature quantities output from each block are normalized using Equation (5).
[0084]
Number
[0085] Loss function L kdft (The fifth loss function) is defined as a combination of loss functions (the fourth loss function) that evaluate the differences in the outputs of each layer. Specifically, the binary-cross entropy loss (L bc ) shown in Equation (6), the L2-loss (L Gk ) of the Gram Matrix values of each block shown in Equation (7), and the MSE error (L ft ) between the outputs of the full-connect layer immediately before the discriminative layer shown in Equation (8) are combined and defined as in Equation (9).
[0086]
Number
[0087]
Number
[0088]
Number
[0089]
Number
[0090] Here, y represents the output (predicted value) of the teacher model, and ŷ represents the output (predicted value) of the student model. Also, λ1, λ3, λ5, λ ft , λ bc represent weights.
[0091] L Gk The introduction of Gk is to more accurately distill the features of the teacher model 901 into the student model 902.
[0092] In Equation (9), each weight, for example, λ1, λ3, λ5 is 0.4, λ ft is 0.04, λ bc is set to 0.01. Also, an example of the parameters during learning is shown in FIG. 10.
[0093] Note that λ1, λ3, λ5, λ ft , λ bc may be changed according to the progress of learning.
[0094] Specifically, as shown in FIG. 14, during the learning process, when the learning error shown on the vertical axis is equal to or greater than the threshold value A (initial stage of learning), the values of λ1, λ3, λ5 are set large so as to match the features of the previous stage of the network. Also, when the learning error becomes equal to or less than the threshold value B (late stage of learning), in order to further improve the accuracy of the prediction result closer to the final layer, λ ft , λ bc values are set large. Also, the weights may be set based on the number of epochs. For example, when it is less than or equal to the threshold value C (initial stage of learning), the values of λ1, λ3, λ5 are set large, and when it is greater than or equal to the threshold value C (late stage of learning), λ ft , λ bc values are set large.
[0095] Furthermore, a method of setting weights by combining the above two may also be used.
[0096] FIG. 11 is a diagram showing an example of the prediction of the teacher model 901 and the student model 902.
[0097] Here, it is assumed that pathological tissue image data is input to the teacher model 901 and MRI data is input to the student model 902. Also, it is assumed that the teacher model 901 and the student model 902 predict the ACU.
[0098] The resolution of the pathological tissue image data is two or more digits higher than that of the MRI data, and the amount of information is large. Here, a model was generated to estimate the disease risk equivalent to the pathological data with a large amount of information using only the MRI data with a small amount of information. The student model (Model_kdft) using the method of this example showed higher discrimination performance compared to the student model (Model_mri) that predicts using only MRI data, and performance close to that of the teacher model (Modek_hst) was obtained.
[0099] FIG. 12 shows the two-dimensional space mapping after dimensionality reduction of the feature vectors (dense2 output) of Model_mri, Model_hist, and Model_kdft using t-SNE (t-distributed Stochastic Neighbor Embedding) (see Non-Patent Document 7).
[0100] In Model_mri, the data of the two classes overlap at the boundary, but it can be seen that Model_hist has a distribution in which the two classes can be classified. Although there is still overlap in Model_kdft, the degree of overlap is smaller than that of Model_mri and the range is also narrower, suggesting that it is closer to the classification performance of Modek_hst.
[0101] According to this example, it is possible to effectively inherit the features of the teacher model in both model construction and actual clinical situations where the performance of a highly accurate prediction model is maintained and the input data is more restricted. That is, a prediction model with high prediction accuracy can be generated even for input data with a limited amount of information.
Example
[0102] Example 3 will explain the framework for realizing the transfer learning described in Example 1 and Example 2. The configuration of the computer system in Example 3 is the same as that in Example 1. Also, the hardware configuration and software configuration of the computer 100 in Example 3 are the same as those in Example 1.
[0103] FIG. 13 is a diagram for explaining the framework of transfer learning in Example 3.
[0104] Computer 100 executes preprocessing 1302 to extract data purification, features of data, and relationships between data for the data stored in database 1301 (public database and company's own database), and generates base model 1303 by executing machine learning using the preprocessed data.
[0105] Starting from the base model, computer 100 executes the transfer learning of Example 1 using in vivo clinical data 1304 to generate a high-precision clinical prediction model (In-vivo Model) 1305.
[0106] When the target of the in vivo clinical data has a restricted amount of input data information compared to the input data information used in the learning of the base model, or when it is desired to generate a model smaller in size than the base model, computer 100 executes the transfer learning of Example 2 to generate a high-precision clinical prediction model (Tiny Model) 1307.
[0107] As a case where a small-sized model is required, a case where there are restrictions on the hardware implementation environment of the computer or computer system using the prediction model is assumed.
[0108] Data purification in preprocessing 1302 refers to data cleaning processes such as data cleansing, missing value interpolation, noise removal such as outliers, and normalization. Feature extraction refers to, for example, the process of identifying important biological pathways by pathway enrichment analysis. From the gene groups included in the gene list, pathways with high enrichment scores are analyzed. Also, extraction processing of partial structures by fingerprinting can be considered. The most commonly used fingerprint is Morgan fingerprint (ECFP) processing. It is based on a two-dimensional structure and is one of the methods for expressing molecular structures. It is mainly used for "similarity search" of molecules and construction of quantitative structure-activity / property relationship (QSAR: Quantitative Structure-Activity Relationship, QSPR: Quantitative Structure-Property Relationship) models.
[0109] In addition, it refers to processes such as extraction of specific features by filtering processing and dimensionality reduction. Relationship extraction between data refers to the process of integrating multiple data in consideration of the relationships between each input data and adding new parameters. For example, there is an idea of using the correlation coefficient between data as input data. Note that this feature extraction and relationship extraction between data are not necessarily required processes.
[0110] With the framework according to this embodiment, it becomes possible to perform effective knowledge transfer and distillation processing from a base model that utilizes a large-scale public database or existing Archive samples to a small number (few features) of clinical samples, and to enhance and streamline the generation of prediction models that meet the on-site environment and requirements in actual clinical practice.
[0111] Note that the present invention is not limited to the above-described embodiments, and various modifications are included. Also, for example, the above-described embodiments explain the configuration in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Further, it is possible to add to, delete from, or replace a part of the configuration of each embodiment with other configurations.
[0112] In addition, each of the above configurations, functions, processing units, processing means, etc. may be realized in hardware by designing a part or all of them, for example, by using an integrated circuit. Also, the present invention can be realized by a program code of software that realizes the functions of the embodiments. In this case, a storage medium recording the program code is provided to a computer, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing it constitute the present invention. As a storage medium for supplying such a program code, for example, a flexible disk, CD-ROM, DVD-ROM, hard disk, SSD (Solid State Drive), optical disk, magneto-optical disk, CD-R, magnetic tape, non-volatile memory card, ROM, etc. are used.
[0113] Also, the program code for realizing the functions described in this embodiment can be implemented in a wide range of programs or script languages such as assembler, C / C++, perl, Shell, PHP, Python, Java (registered trademark), etc.
[0114] Furthermore, by distributing the program code of software that realizes the functions of the embodiments via a network, it can be stored in a storage means such as a hard disk or memory of a computer or a storage medium such as a CD-RW or CD-R, and a processor included in the computer reads and executes the program code stored in the storage means or the storage medium.
[0115] In the above embodiments, the control lines and information lines show those considered necessary for explanation, and not necessarily all the control lines and information lines on the product. All components may be interconnected with each other.
Explanation of Signs
[0116] 100 Computer 101 CPU 102 Main Memory Device 103 Auxiliary Memory Device 104 Network Adapter 105 Input Device 106 Output Device 108 Internal Bus 110 Information Terminal 111 External Memory Device 200 First Learning Unit 201 Second Learning Unit 202 Prediction Unit 210 First Learning Data DB 211 Second Learning Data DB 212 Model DB 220 Input Data 221 Prediction Result 401, 402 Network 901 Teacher Model 902 Student Model 903, 904 Input Data 1500 GUI
Claims
1. A computer system comprising a processor, a storage device connected to the processor, and a computer having a connection interface connected to the processor, A base model is generated by machine learning using the first training dataset stored in the aforementioned storage device. A second model is generated by transfer learning using the first model and the second training dataset stored in the storage device. The computer system is characterized in that, in the transfer learning described above, the second model is generated using at least one of the following: a third loss function which includes a second loss function that adjusts the influence of the learning data on the learning data according to the distribution characteristics of the items corresponding to the prediction target of the second model included in the learning data constituting the second learning dataset; and a fifth loss function which includes a fourth loss function that evaluates the difference between the outputs of the first model and the second model.
2. A computer system according to claim 1, The computer system is characterized in that the second loss function is a function for suppressing updates of the second model by training data in which the values of items corresponding to the prediction target of the second model, which are included in the training data constituting the second training dataset, do not fall within a predetermined range.
3. A computer system according to claim 1, The first and second models are networks including an input layer, a plurality of hidden layers, and an output layer, The computer system is characterized in that the fourth loss function includes a loss function that evaluates the difference between the outputs of at least one layer of the input layer and the plurality of intermediate layers of the first model and the second model, and a loss function that evaluates the difference between the outputs of the output layer of the first model and the second model.
4. A computer system according to claim 1, The third loss function and the fifth loss function include coefficients, The computer system is characterized in that it performs the transfer learning while changing the coefficients included in the third loss function or the fifth loss function according to the progress of the transfer learning.
5. A computer system according to claim 1, The second model is a computer system characterized by being a model that makes predictions about a patient's disease or condition.
6. A computer system according to claim 1, The training data included in the first training dataset includes genetic information of cell lines and drug responses. The first model outputs the drug response from the genetic information of the cell line, The training data included in the second training dataset is clinical data showing the patient's drug response. The second model is a computer system characterized by accepting a patient's gene mutation information as input and making predictions regarding the patient's drug response.
7. A method for learning a model executed by a computer system, The computer system comprises a computer having a processor, a storage device connected to the processor, and a connection interface connected to the processor. The learning method for the aforementioned model is: The first step is for the processor to generate a first model by performing machine learning using a first training dataset stored in the storage device, and to store the first model in the storage device. The processor includes a second step of generating a second model by performing transfer learning using the first model and the second training dataset stored in the storage device, and storing the second model in the storage device. A method for learning a model, characterized in that the second step includes the step of the processor generating the second model using at least one of the following: a third loss function which includes a second loss function which adjusts the influence of the learning data on the learning data according to the distribution characteristics of the items corresponding to the prediction targets of the second model included in the learning data constituting the second learning dataset; and a fifth loss function which includes a fourth loss function which evaluates the difference between the outputs of the first model and the second model.
8. A method for learning the model described in claim 7, A model training method characterized in that the second loss function is a function for suppressing updates of the second model by training data in which the values of items corresponding to the prediction target of the second model, which are included in the training data constituting the second training dataset, do not fall within a predetermined range.
9. A method for learning the model described in claim 7, The first and second models are networks including an input layer, a plurality of hidden layers, and an output layer, A model learning method characterized in that the fourth loss function includes a loss function that evaluates the difference between the outputs of at least one layer of the input layer and the plurality of intermediate layers of the first model and the second model, and a loss function that evaluates the difference between the outputs of the output layer of the first model and the second model.
10. A method for learning the model described in claim 7, The third loss function and the fifth loss function include coefficients, A method for learning a model, characterized in that the second step includes the step of the processor performing the transfer learning while changing the coefficients included in the third loss function or the fifth loss function according to the progress of the transfer learning.
11. A method for learning the model described in claim 7, The second model is a model that makes predictions about a patient's disease or condition, and is a method for learning such a model.
12. A method for learning the model described in claim 7, The training data included in the first training dataset includes genetic information of cell lines and drug responses. The first model outputs the drug response from the genetic information of the cell line, The training data included in the second training dataset is clinical data showing the patient's drug response. The second model is characterized by being a model that accepts a patient's gene mutation information as input and makes predictions regarding the drug response in the patient.
13. A computer system comprising a processor, a storage device connected to the processor, and a computer having a connection interface connected to the processor, The memory device stores the second model generated by transfer learning of the base first model, The first model is generated by machine learning using the first training dataset, The second model is generated by the transfer learning using the first model and the second training dataset. In the transfer learning described above, at least one of the following is used: a third loss function that includes a second loss function that adjusts the influence of the learning data on the learning data according to the distribution characteristics of the items corresponding to the prediction target of the second model included in the learning data constituting the second learning dataset; and a fifth loss function that includes a fourth loss function that evaluates the difference between the outputs of the first model and the second model. The aforementioned computer system, Accepts input data, A computer system characterized by performing predictions using the second model and the input data.
14. A method of information processing performed by a computer system, The aforementioned computer system, A computer having a processor, a storage device connected to the processor, and a connection interface connected to the processor, The memory device stores the second model generated by transfer learning of the base first model, The first model is generated by machine learning using the first training dataset, The second model is generated by the transfer learning using the first model and the second training dataset. In the transfer learning described above, at least one of the following is used: a third loss function that includes a second loss function that adjusts the influence of the learning data on the learning data according to the distribution characteristics of the items corresponding to the prediction target of the second model included in the learning data constituting the second learning dataset; and a fifth loss function that includes a fourth loss function that evaluates the difference between the outputs of the first model and the second model. The aforementioned information processing method is The processor takes the step of receiving input data, An information processing method characterized by comprising the step of the processor making a prediction using the second model and the input data.