Prediction method, system, and program for predicting quantitative index in positivity of event caused by chemical material
The information processing system uses machine learning to predict EC3 values for skin sensitization by avoiding overfitting and setting application limits, achieving high accuracy in EC3 value predictions.
Patent Information
- Application Number
- JP2023081119
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-16
- Publication Date
- 2024-11-28
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing in silico evaluation systems for predicting the EC3 value of skin sensitization require additional verification using external data and have limitations in accuracy.
An information processing system that uses machine learning to construct predictive models for EC3 values without external verification, employing a distribution processing unit to separate data into internal and external verification sets, and utilizes CatBoost models to avoid overfitting, incorporating ITSv2 hazard evaluation as a threshold for applicability.
The system achieves accurate prediction of EC3 values with improved accuracy, demonstrated by R2 values of 0.862 for training data, 0.815 for internal verification, and 0.824 for external verification, significantly reducing overfitting and enhancing prediction reliability.
Smart Images

Figure 2024165190000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a method, system, and program for predicting a quantitative indicator of a positive event caused by a chemical substance using a machine learning model. [Background technology]
[0002] In the evaluation of skin sensitization of chemical substances contained in raw materials and products for industrial products, the presence or absence of skin sensitization and the concentration at which sensitization occurs are evaluated using animals before human testing, and finally safety is evaluated through human testing such as patch tests and usage tests. However, in addition to considerations of animal welfare and costs, the 2013 EU directive banning animal testing for cosmetics and the sale of cosmetics that have been tested on animals has led to a strong demand for the development of alternative testing methods that do not use animals.
[0003] There are three non-animal testing methods for assessing the presence or absence of skin sensitization, which examine and evaluate biological reactions (KE (Key Event)) based on adverse outcome pathways (AOP: Adverse Outcome Pathway). Specifically, the DPRA (Direct Peptide Reactivity Assay) method corresponds to KE1 (covalent binding to skin proteins), the KeratinoSens (KE2) method corresponds to KE2 (induction of inflammatory cytokines and induction of cell defense gene pathways), and the KE3 (KE4) method corresponds to KE5 (induction of inflammatory cytokines and induction of cell defense gene pathways). TM In chemico / in vitro test methods such as the human cell line activation test (h-CLAT) method corresponding to KE3 (induction of inflammatory cytokines and surface antigens, migration of dendritic cells) are used.
[0004] In contrast, there has been no sufficient evaluation of the concentration at which sensitization occurs without using animals. The EC3 value of the local lymph node test (LLNA) is used in human risk assessment of skin sensitization as the standard of the United Nations Globally Harmonized System of Classification and Labeling of Chemicals (UNGHS) and as the No Expected Sensitization Induction Level (NESIL), which is an index equivalent to the level of no observable adverse effects. If the EC3 value of this LLNA can be quantitatively predicted, it will be possible to predict the risk of skin sensitization in more detail before human evaluation. However, the in silico evaluation system proposed to quantitatively predict the EC3 value so far requires further verification using external data not included in the model construction, and has limited accuracy (see Non-Patent Documents 1 and 2). [Prior art documents] [Patent documents]
[0005] [Non-Patent Document 1] M. Hirota, et al., Development of an artificial neural network model for risk assessment of skin sensitization using human cell line activation test, direct peptide reactivity assay, KeratinoSensTM and in silico structure alert parameter. J. Appl. Toxicol., 38 (2018), p514-526 [Non-Patent Document 2] K. Ambe et al., Development of quantitative model of a local lymph node assay for evaluating skin sensitization potency applying machine learning CatBoost. Regulatory ToxicoLogy and Pharmacology, 125 (2021) 105019 Summary of the Invention [Problem to be solved by the invention]
[0006] In view of the above-mentioned situation, the present invention aims to provide an in silico evaluation system that can quantitatively and accurately predict positive indicators of events caused by chemical substances, such as EC3 values, without additional verification using external data. [Means for solving the problem]
[0007] That is, the present invention includes the following inventions. (1) An information processing system for predicting a quantitative indicator of a positive event caused by a chemical substance, the information processing system comprising: a response variable memory unit that stores information on a quantitative indicator of a positive event caused by a chemical substance as a response variable; an explanatory variable memory unit that stores at least one piece of information selected from the group consisting of experimental information, information on a chemical structure, and information on a substance similar to a test substance as an explanatory variable; an allocation processing unit that allocates and stores each piece of information on the response variable and the explanatory variables as learning data and verification data; a prediction model construction processing unit that uses the response variable and explanatory variables of the learning data by a machine learning technique to construct a prediction model of the value of the indicator, evaluates the prediction model using the response variable and explanatory variables of the verification data, and reconstructs the prediction model as necessary; and an output processing unit that outputs the predicted value of the indicator of the chemical substance using the prediction model.
[0008] (2) The information processing system according to (1), wherein the predictive model construction processing unit uses the objective variable and explanatory variables for the validation to evaluate the predictive accuracy of the predictive model and the presence or absence of overlearning.
[0009] (3) The information processing system according to (2), wherein the prediction model construction processing unit evaluates the prediction accuracy and determines whether or not overlearning occurs based on an R value and an RMSE value calculated by the following formulas.
[0010]
number
[0011] There are no clear rules for the R2 value, but it is considered good to have an R2 value of 0.8 or more. RMSE is the square root of the root mean square of the predicted value minus the actual value. It is used to compare training data and validation data, and the smaller the value, the better. For example, if the R2 value is "0.889" for the training data and "0.372" for the validation data, the model is evaluated as having overfitted and in need of improvement.
[0012] (4) The information processing system of (3), wherein the allocation processing unit further allocates the validation data to two types: internal validation data and external validation data, and the predictive model construction processing unit uses the internal validation data to determine whether or not overfitting has occurred, and uses the external validation data to evaluate prediction accuracy.
[0013] (5) The information processing system according to (1), wherein the allocation processing unit compresses each piece of information about the objective variables and explanatory variables by normalizing and standardizing the information to match the scale (number of digits), and then allocates the information to the learning data and the validation data and stores the data.
[0014] (6) The information processing system according to (1), wherein the predictive model construction processing unit uses a result of a correct / incorrect ITSv2 hazard assessment (OECD Guideline No. 497 ITSv2) as a threshold for the range of application.
[0015] (7) A control program for causing a computer to function as the information processing system described in (1), the control program causing a computer to function as the allocation processing unit, the predictive model construction processing unit, and the output processing unit.
[0016] (8) An information processing method executed by a system which has an objective variable memory unit which stores information on a quantitative indicator of a positive event caused by a chemical substance as an objective variable, and an explanatory variable memory unit which stores at least one of experimental information, information on a chemical structure, and information on a substance similar to a test substance as an explanatory variable, and which predicts a quantitative indicator of a positive event caused by a chemical substance, the information processing method including: a classification step of classifying and storing information on the objective variable and the explanatory variable into learning data and validation data; a prediction model construction step of constructing a prediction model of the value of the indicator using the objective variable and explanatory variables of the learning data by a machine learning technique, evaluating the prediction model using the objective variable and explanatory variables of the validation data, and reconstructing the prediction model as necessary; and an output step of outputting the predicted value of the indicator of the chemical substance using the prediction model. Effect of the Invention
[0017] According to the present invention as described above, it is possible to provide an in silico evaluation system that can quantitatively and accurately predict the positive indicator values of events caused by chemical substances without additional separate verification using external data.
[0018] Specifically, as shown in the following examples, 195 substances with in chemico / in vitro test data on each key event of LLNA EC3 value and skin sensitization AOP (Adverse Outcome Pathway) were used, and a machine learning model using each test data, molecular descriptors, and distance information based on the read-across concept was combined with ITSv2 hazard assessment to create a prediction system that combined two CatBoost models and ITSv2 hazard assessment. The performance of the prediction system was 0.862 for training data, 0.550 for internal validation data, and 0.617 for external validation data in terms of R2 value. Furthermore, substances that were misclassified in the ITSv2 hazard assessment were considered to have properties that do not match the correspondence between the assumed AOP and skin sensitization, and the correctness of the ITSv2 hazard assessment was set as the threshold for the scope of application of this prediction system. When the ITSv2 hazard assessment was re-evaluated excluding substances with incorrect ITSv2 hazard assessment, the prediction accuracy was significantly improved, with R2 values of 0.963 for training data, 0.681 for internal validation data, and 0.815 for external validation data. In addition, by reconstructing the CatBoost model using only substances within the scope of application, the prediction accuracy was further improved, with R2 values of 0.995 for training data, 0.787 for internal validation data, and 0.824 for external validation data. [Brief description of the drawings]
[0019] [Figure 1] 1 is a block diagram showing a schematic configuration of an information processing system according to a representative embodiment of the present invention. [Diagram 2] Plot of observed vs. predicted values using Model R4 when only NS substances were included in the external validation data. [Diagram 3] An explanatory diagram showing the workflow of the final prediction system, Model F1. [Figure 4] Plot of actual values vs. predicted values using the final prediction system model. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0020] Next, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.
[0021] The information processing system according to the present invention is an information processing system that performs quantitative prediction of skin sensitization by a computer, and is composed of one or more information processing devices including a processing device 2, a storage means 3, an information display unit 4, and an input means 5, as shown in Fig. 1. Specifically, the information processing system is a computer device that includes a processing device 2, a storage means 3, an input means 5 such as a pointing device, a keyboard, or a touch panel, an information display unit 4 such as a display, and other communication control units (not shown). Note that, in this embodiment, a system that performs quantitative prediction of skin sensitization is described as an example, but the present invention is not limited thereto, and can be used as a system that can define positive and negative and quantitatively predict indicators of positive events caused by chemical substances, such as other toxicities such as repeated dose toxicity and metabolic reactivity, and can be used for a wide range of biological reactions.
[0022] The processing device 2 is mainly composed of a CPU such as a microprocessor, and has a storage unit consisting of RAM and ROM (not shown) in which programs and processing data that define the procedures of various processing operations are stored. The storage means 3 is composed of a memory or a hard disk inside or outside the information processing system 1. A part or all of the contents of the storage unit may be stored in a memory or a hard disk of another computer communicatively connected to the information processing system 1. The storage means 3 has a response variable storage unit 31 that stores information on the LLNA EC3 value as an index of skin sensitization of a plurality of chemical substances as a response variable, and an explanatory variable storage unit 32 that stores at least one of information on the KE test results for the plurality of chemical substances, information on chemical structures, and information on similar substances as explanatory variables.
[0023] The objective variables (LLNA EC3 value information) can be collected from the websites of public institutions and papers. As a specific example, information on 195 substances can be obtained from the following two data sources. (Data Source 1) Development of an artificial neural network model for risk assessment of skin sensitization using human cell line activation test, direct peptide reactivity assay, KeratinoSens TM and in silico structure alert parameter, M. Hirota(Shiseido Co. Ltd) et al., J Appl Toxicol. 2018; 38: 514-526. (Data source 2) Development of quantitative model of a local lymph node assay for evaluating skin sensitizasion potency applying machine learning CatBoost, K. Ambe et al., Regulatory Toxicology and Pharmacology, 125 (2021) 105019.
[0024] The explanatory variables are, for example, the descriptors obtained by molecular descriptor calculation software for the substance from which the objective variable was obtained, and the Log(EC150(CD86)(μM)), Log(EC200(CD54)(μM)), Log(MIT(μM)), Log(CV75(μM)), Log(Adjusted Cys depletion), Log(Adjusted Lys depletion), Log(KEC1.5), h-CLAT P / N, CD86 P / N, CD54 P / N, DPRA P / N, KeratinoSens obtained from the literature. TMIn the Euclidean space formed by the data such as P / N and the data normalized based on the training data, the shortest distance between each data and the substances classified as NS, Weak, Moderate, Strong, and Extreme in the training data ("min_dist_NS", "min_dist_Weak", "min_dist_Moderate", "min_dist_Strong", "min_dist_Extreme") and the Log(EC3(μmol / cm) of the nearest neighbor substances are calculated. 2 )) and Log(EC3 value ("Nearest Log(EC3(μmol / cm 2 )" and "Nearest Log(EC3 value)" are examples.
[0025] Functionally, the processing device 2 comprises an allocation processing unit 21 that allocates and stores information on the response variables and explanatory variables as learning data and validation data, a prediction model construction processing unit 22 that uses a machine learning technique to construct a prediction model for the EC3 value using the response variables and explanatory variables of the learning data, evaluates the prediction model using the response variables and explanatory variables of the validation data, and performs necessary reconstruction of the prediction model, and an output processing unit 23 that uses the prediction model to output the predicted EC3 value of a chemical substance, and these processing functions are realized by the above-mentioned program.
[0026] The verification data allocated by the allocation processing unit 21 is test data for evaluating whether the constructed prediction model can be applied to unknown data. When allocating the learning data and the verification data, it is preferable to perform a process of compressing the data to 0 to 1 by normalization and standardization to match the scale. This allows data with different digits to be handled with the same calculation formula. Furthermore, the allocation processing unit 21 further allocates the verification data to two types, that is, internal verification data and external verification data. Specifically, the information on the objective variable and the explanatory variable is compressed by normalization and standardization to match the scale (number of digits), and then allocated to the learning data and the verification data for storage.
[0027] The prediction model construction processing unit 22 constructs a model based on the learning data, avoids overlearning by adopting a model that fits well to the internal validation data, and objectively evaluates the prediction accuracy using the external validation data. The prediction model can be constructed using ANN, Read-across, etc. Among them, the CatBoost model is a GBDT algorithm published in 2017 that handles categorical variables well using a permutation-driven method and is less prone to over-fitting. In the present invention, it is preferable to use CatBoost because several categorical variables are handled as explanatory variables.
[0028] Furthermore, the prediction model construction processing unit 22 is configured to avoid overlearning by combining the ITSv2 hazard assessment among the skin sensitization assessment methods (Defined Approach) listed in the guideline for skin sensitization assessment methods (OECD Test Guideline No. 497), and re-evaluating the results of the evaluation as the threshold of the application range, excluding substances that were incorrectly assessed. As a method other than the ITSv2 hazard assessment, it is also possible to use the 2 out of 3 approach or the 3 out of 3 approach, which are also widely known as Defined Approach. The prediction model construction processing unit 22 evaluates the prediction accuracy of the prediction model and the presence or absence of overlearning using the objective variable and explanatory variable for verification. Specifically, the prediction accuracy is evaluated and the presence or absence of overlearning is determined by the R2 value and RMSE value obtained by the following formula.
[0029]
number
[0030] There are no clear rules for the R2 value, but a value of 0.8 or more is considered good. RMSE is the square root of the root mean square of the predicted value minus the actual value. It is used to compare training data and validation data, and the smaller the value, the better. For example, if the R2 value is 0.889 for the training data and 0.372 for the validation data, the model is evaluated as having overfitted and in need of improvement. The presence or absence of overfitting is determined using internal validation data. Prediction accuracy is evaluated using external validation data.
[0031] The output processing unit 23 performs processing such as displaying the results of the predicted EC3 values of the chemical substances on a display or the like, and transmitting the results to a client computer connected for communication.
[0032] Although the embodiments of the present invention have been described above, the present invention is not limited to these embodiments. For example, instead of configuring the processing device with software processing by a computer, it is preferable to configure part or all of it with a hardware processing circuit. In this case, an artificial intelligence processing circuit can be used as the machine learning mechanism. It goes without saying that the present invention can be embodied in various forms without departing from the gist of the present invention. EXAMPLES
[0033] Hereinafter, an information processing system according to the present invention will be described based on a specific embodiment.
[0034] (Obtaining the target variable) In the above mentioned data sources 1 and 2, for duplicate substances with the same compound name, CAS No., or SMILES, data from data source 2, which cites a newer paper as a database, will be used. After excluding duplicates, 195 substances will be analyzed. In addition to the LLNA EC3 values for these multiple chemical substances, Log(EC150(CD86)(μM)), Log(EC200(CD54)(μM)), Log(MIT(μM)), Log(CV75(μM)), Log(Adjusted Cys depletion), Log(Adjusted Lys depletion), Log(KEC1.5), h-CLAT P / N, CD86 P / N, CD54 P / N, DPRA P / N, KeratinoSens TM Obtain P / N data (P / N indicates Positive / Negative, and Log is common logarithm).
[0035] In addition, substances that are outside the scope of each test used in ITSv2 (OECD Guideline No. 497 ITSv2) hazard assessment (e.g., substances with negative h-CLAT results and LogP > 3.5) are excluded from the analysis, leaving 195 substances to be analyzed (OECD, 2022b). ITSv2 is an assessment method for identifying the skin sensitization hazards (UN GHS NC (Not Classified) vs. Category 1) and risks (UN GHS NC vs. Sub-Category 1B vs. Sub-Category 1A) of chemicals without animal testing, and is included in OECD Guideline No. 497 (OECD, 2021). In addition to the two test methods corresponding to the skin sensitization AOP (DPRA, OECD TG 442C; h-CLAT, OECD TG 442E), the assessment is based on the predicted values by the OECD QSAR Toolbox (ver.4.5).
[0036] In addition, to organize the data, the EC3 (%) of LLNA Category NS (EC3 (%) > 100) is unified to "150" (Ambe et al., 2021). In reality, the EC3 value cannot be determined because there is no sensitizing potential, but "150" is set as the upper limit for regression. Similarly, the EC3 (%) of LLNA Category Extreme (EC3 (%) < 0.1) is unified to "0.1". Since the common logarithm of the EC3 value is taken as the objective variable for regression, small values after the decimal point are converted to large negative values. Since theoretically it is possible for the value to reach negative infinity, a lower limit for EC3 (%) is set.
[0037] For EC150 (CD86) (μM) and EC200 (CD54) (μM), substitute "100000" for each in the case of negative. Therefore, the maximum value of Log(MIT(μM)) is "5". For Adjusted Cys / Lys depletion, the minimum value is set to "0.01" to satisfy the real number condition when taking logarithmic values. For KEC1.5, the maximum value is unified to "2000". All 195 substances are sorted in descending order by EC3 (%) and numbered consecutively from S001 to 195.
[0038] (Getting explanatory variables) Using the molecular descriptor calculation software Mordred, molecular descriptors are obtained for the 195 substances mentioned above. Descriptors that can be calculated for all substances and do not show the same value for all substances are obtained. Among them, the Log(EC3(μmol / cm 2 ) and not highly correlated with each other, were designated as "Molecular Descriptor Set 1." In addition, the Log(EC3(μmol / cm) of the substances with skin sensitization not classified as NS in the learning data was designated as "Molecular Descriptor Set 2." 2 )) and not highly correlated with each other are called "molecular descriptor set 2" and are used as explanatory variables "Molecular descriptor 1, 2" (hereinafter referred to as "MD1" and "MD2").
[0039] In addition, the data obtained from the literature are Log(EC150(CD86)(μM)), Log(EC200(CD54)(μM)), Log(MIT(μM)), Log(CV75(μM)), Log(Adjusted Cys depletion), Log(Adjusted Lys depletion), Log(KEC1.5), h-CLAT P / N, CD86 P / N, CD54 P / N, DPRA P / N, KeratinoSens TM P / N is defined as an in vitro / chemico descriptor (hereinafter referred to as "IVD").
[0040] In the Euclidean space formed by the data normalized based on the training data for IVD, MD1, and MD2, the shortest distance between each data and the substances classified as NS, Weak, Moderate, Strong, or Extreme in the training data ("min_dist_NS", "min_dist_Weak", "min_dist_Moderate", "min_dist_Strong", "min_dist_Extreme") and the Log(EC3(μmol / cm) of the nearest neighbor substances were calculated. 2 )) and Log(EC3 value ("Nearest Log(EC3(μmol / cm 2 The seven variables, IVD, MD1, MD2, DD1, and DD2, are used as explanatory variables, IVD, MD1, MD2, DD1, and DD2, respectively.
[0041] (Distribution process) In the skin sensitization classification, LLNA Category NS (EC3 value >100) is LLNA Negative, and the rest (EC3 value ≦100) is LLNA Positive, and these are used as the objective variables. In the regression model, Log(EC3 value) and Log(EC3(μmol / cm 2 ) is used as the objective variable. The EC3 value is calculated by (EC3 value)*250 / MW, and the EC3 value is calculated as EC3 (μmol / cm2 ) can be converted into EC3 values. In order to distribute data uniformly by the distribution processing unit 21, the EC3 values are numbered 1 to 5 from the top, with 1, 3, and 5 set for learning, 2 for internal validation, and 4 for external validation.
[0042] (Predictive model construction) The model construction by the prediction model construction processing unit 22 uses Python for Windows v3.9.12 and can use CatBoostRegressor for CatBoost (1.0.6). For parameter tuning, it is preferable to use the parameter tuning tool optuna (2.10.1) based on the Bayesian optimization algorithm and tune "depth", "learning_rate", "bagging_temperature", and "od_type".
[0043] Using the molecular descriptor calculation software Mordred, 703 molecular descriptors that could be calculated for all 195 substances and had a non-zero variance were obtained, and MD1 and MD2 were selected based on the following criteria. The selection criteria for MD1 were Log(EC3(μmol / cm) of all training data. 2 )) is 0.20 or more, and the absolute value of the correlation coefficient between the descriptors is 0.75 or less. The selection criteria for MD2 are the Log(EC3(μmol / cm 2 )) have an absolute value of 0.30 or more in correlation coefficients, and the absolute value of the correlation coefficients between the descriptors themselves is 0.75 or less. Based on the above criteria, 10 descriptors were selected as MD1 and 9 descriptors were selected as MD2, as shown in Table 1.
[0044] [Table 1]
[0045] DD1 and DD2 were calculated based on IVD, MD1, and MD2, respectively.
[0046] Next, a regression model was constructed using IVD. Parameter tuning for each model was performed using optuna, a parameter tuning tool based on a Bayesian optimization algorithm. In addition, the EC3 value of sensitizing NS substances was given an upper limit of "150". Since this is not an actual measured value, it may have a negative effect on the prediction of continuous values. Therefore, whether or not to include NS substances in the data was set as one of the conditions. The effects of the objective variable and NS data were confirmed using the R2 value and RMSE as evaluation indices. The results are shown in Table 2. In particular, the R2 value was higher when NS substances were included in the external validation data. This is thought to be because NS substances are more likely to show high values by expanding the range of possible EC3 values.
[0047] [Table 2]
[0048] For non-NS substances, predictions by Models R1 and R2 were compared by calculating evaluation indices as Models R1' and R2'. The results are shown in Table 3. Models R3 and R4 had prediction accuracy equal to or higher than R1' and R2'.
[0049] [Table 3]
[0050] From the above, it was shown that excluding substances with skin sensitizing properties is effective in quantitative prediction. In addition, a comparison of Models R1 and R2, and R3 and R4 showed that setting Log(EC3(μmol / cm)) as the objective variable was more effective than setting Log(EC3(%)). 2 )) showed better values for most of the evaluation indices, and Log(EC3(μmol / cm 2 ) was shown to be effective as a response variable. Therefore, the response variable of the subsequent regression models was Log(EC3(μmol / cm 2 )) was decided.
[0051] where Log(EC3(μmol / cm 2 )) as the objective variable, and Model R4, which was constructed using data excluding NS substances, predicts NS substances with values lower than the actual measured values, as shown in Figure 2, because it is not possible to learn NS substances. Since it is not possible to know whether a substance is NS or not at the prediction stage, it is difficult to obtain a model that is effective for a variety of data including NS substances using a single regression model obtained from data only on substances with positive skin sensitization. However, conversely speaking, if substances with positive skin sensitization can be identified using a skin sensitization hazard assessment method, a model such as Model R4 can be applied only to substances with positive skin sensitization. Therefore, we decided to incorporate the ITSv2 hazard assessment, which is a highly reliable method listed in the OECD Guideline No. 497, into the prediction system.
[0052] Next, the influence of MD (Molecular descriptor) and DD (Distance-based descriptor) was evaluated. The results are shown in Table 4. When data on NS substances was included, MD1 and DD1, which were selected under the condition that NS substance data was included, were adopted. Similarly, when data on NS substances was not included, MD2 and DD2 were adopted. A comparison of Models R5 and R6, and R7 and R8 in Table 4 showed that both MD and DD were effective explanatory variables for prediction.
[0053] [Table 4]
[0054] The final LLNA EC3 predicted value was calculated as the average of the predicted value of Model R6 and the value equivalent to negative, Log(150*250 / MW), if the ITSv2 hazard assessment was negative, and was calculated as the average of the predicted value of Model R6 and the predicted value of Model R8 if the ITSv2 hazard assessment was positive, as shown in Figure 3. The results of evaluating this prediction system as Model F1 are shown in Table 5 and Figure 4.
[0055] In addition, in the process up to this point, the prediction accuracy for the internal validation data was significantly poor. Looking at the plot of actual values vs. predicted values for Model F1 in Figure 4, there are substances for which the predictions are clearly off in the internal validation data, and many of these are substances that were incorrectly classified in the ITSv2 hazard assessment. These substances are thought to be substances with properties that do not match the correspondence between each test expressing the skin sensitization AOP assumed in the ITSv2 hazard assessment and skin sensitization.
[0056] Therefore, Model F1' was evaluated by excluding substances for which the ITSv2 hazard assessment was false positive / negative from each data set. The agreement rate of the ITSv2 hazard assessment for all 195 substances was 0.790, and the balanced accuracy was 0.725.
[0057] In addition, substances with false positive / negative predictions in the ITSv2 hazard assessment were removed from each data set, and Models R6' and 8' corresponding to Models R6 and 8 were reconstructed and incorporated into Model F1 to be evaluated as Model F2. In Models R6' and R8', MD1' and 2' selected using the same method as MD1 and 2 were used as MD to calculate DD1' and 2'.
[0058] [Table 5]
[0059] As can be seen from Table 5, Model F1 had the lowest prediction accuracy for the internal validation data, with an R2 value of 0.550 and an RMSE of 0.639. For the external validation data, the R2 value was 0.617 and the RMSE was 0.637. For Model F1', the R2 value was 0.681 and the RMSE was 0.537 for the internal validation data, and the R2 value was 0.815 and the RMSE was 0.441 for the external validation data. By excluding substances that were misclassified in the ITSv2 hazard assessment from the model's scope, overfitting was reduced, and the prediction accuracy was significantly improved. For Model F2, which reconstructed the CatBoost model using only substances within the scope of application, the R2 value was 0.824 and the RMSE was 0.430 for the external validation data, exceeding the prediction accuracy of Model F1 and Model F1'. This is believed to be because by limiting the list to substances that are considered to conform to the correspondence between each test expressing the AOP envisaged in ITSv2 and the LLNA Positive / Negative, it was possible to make clearer quantitative predictions.
[0060] In addition, when Model R6 was removed from Model F1, the prediction accuracy was lower than that of the original Model F1, with the R2 value of 0.670 for the training data, 0.542 for the internal validation data, and 0.548 for the external validation data (data not shown). This result indicated that Model R6 has the effect of mitigating the impact of misclassification due to ITSv2 hazard evaluation. Thus, excluding Model R6 increases the load of ITSv2 hazard evaluation, excluding ITSv2 hazard evaluation does not allow the reinforcement of prediction accuracy for positive substances by R8 to function well, and excluding Model R8 weakens the prediction of positive substances. This prediction system is configured to distribute the load by combining three prediction models in a well-balanced manner.
[0061] In accordance with the UN GHS classification, the predicted value was classified into three classes: if it was greater than 100 when converted to EC3 (%) or if it was negative according to the ITSv2 hazard assessment, it was classified as "NC"; if the EC3% was between 2 and 100, it was classified as Subcategory "1B"; and if it was less than 2, it was classified as Subcategory "1A." The results of the evaluation are shown in Table 6 (United Nations. Economic Commission for 2021).
[0062] [Table 6]
[0063] As can be seen from Table 6, the classification using Model F2 had a concordance rate of 0.879 even for the external validation data, which had the lowest prediction accuracy, and sufficient prediction accuracy was obtained. In addition, classification using the predicted values of Model F1 had prediction accuracy equal to or higher than that of ITSv2 risk assessment. In other words, the concordance rate of ITSv2 risk assessment was 60-70% for each dataset, confirming that it is a useful risk assessment method that shows prediction accuracy close to that of machine learning models.
[0064] In addition, the results of calculating the variable importance for Models R6, R8, R6', and R8' are shown in Table 7.
[0065] [Table 7]
[0066] In addition, in Table 7, Log(Adjusted Cys depletion), Log(CV75(μM)), Log(EC150 (CD86)(μM)), Log(Adjusted Lys depletion), Log(MIT(μM)), DPRA P / N, Log(EC200(CD54)(μM)), Log(KEC1.5) are IVD, MATS2dv, ATSC6dv, AATS0v, GATS2dv, GATS1m, ZMIC5, piPC6, NaasC, BalabanJ, VSA_EState8, ZMIC2, ATSC8v are MD, Nearest Log(EC3(μmol / cm 2 )), min dist Weak, min dist Moderate, and min dist NS are DD.
[0067] As shown in Table 7, Log(CV75(μM)) and Log(MIT(μM)) were used as the top 10 variables in all models. These are variables related to h-CLAT, and the results are consistent with previous reports that h-CLAT is important for quantitatively predicting EC3 values (Ambe et al., 2021; Hirota et al., 2018). In addition, for molecular descriptors, variables such as GATS2dv (geary coefficient of lag 2 weighted by valence electrons) and MATS2dv (moran coefficient of lag 2 weighted by valence electrons) were seen at the top. These are variables that express spatial autocorrelation weighted by valence electrons and express the distribution of valence electrons in chemical structures.
[0068] (Consideration) In the above-described examples, an in silico model was constructed to quantitatively predict LLNAEC3 values based on the IATA approach using the in vitro / chemico descriptor, which is test data corresponding to each KE of skin sensitization AOPs, the molecular descriptor calculated based on chemical structure, and the distance-based descriptor based on the read-across concept. In addition, the usefulness of incorporating classification based on ITSv2 hazard assessment in the prediction model and setting the accuracy of classification based on ITSv2 hazard assessment as the threshold for the applicability of the model was demonstrated.
[0069] All 195 substances were sorted in descending order of EC3 (%) and divided into training, internal validation, and external validation data. This method was used to make the distribution of skin sensitization intensity in each data set as uniform as possible. Overfitting was avoided by using the internal validation data for parameter tuning, and the prediction accuracy was objectively evaluated using the external validation data. Cross-validation was not adopted because it made it easier to compare the evaluation results of each prediction model for each data set. It was possible to easily confirm the classification possibility and overfitting of each data set.
[0070] Adopting ITSv2 as a skin sensitization hazard assessment method has the great advantage of being reliable, in addition to having data for implementation. ITSv2 is listed in OECD Guideline No. 497, and the agreement rate with LLNA Positive / Negative is published as 88%, and the balanced accuracy is published as 80% (OECD, 2021). For the data in this example, the agreement rate for all 195 substances was "0.790", and the balanced accuracy was "0.725", showing high prediction accuracy, even if it did not reach the published value.
[0071] In an in silico prediction model, it is a difficult task to ensure the reliability of predictions for a wide range of chemical substances. In the present invention, objective verification is performed by setting external validation data, and setting appropriate application limits is also one of the countermeasures. In this example, the substance with an incorrect ITSv2 hazard assessment is considered to be a substance with properties that do not match the correspondence between each test expressing the skin sensitization AOP assumed in ITSv2 and skin sensitization, and the correctness of the ITSv2 hazard assessment was set as the threshold of the application limit.
[0072] This has greatly improved the prediction accuracy, and as mentioned above, Model F1' showed an R2 value of 0.815 and an RMSE of 0.441 in the external validation data. Since the published agreement rate of ITSv2 is 88% and the current data is 79%, the scope of application is not too narrow and can be a threshold that allows predictions for many substances. However, whether the ITSv2 hazard assessment is correct cannot be determined in advance because the LLNA results are not known at the prediction stage. Therefore, when actually evaluating substances with unknown LLNA results, it is necessary to develop a separate method to determine compatibility with the ITSv2 hazard assessment. For example, it has been reported that negative judgments for chemicals with LogP>3.5 are unreliable in h-CLAT, and substances outside the scope of application of such tests are specified as outside the scope of application of ITSv2 (OECD., 2021). Compatibility will need to be considered comprehensively, not just in terms of the scope of application of each test.
[0073] In the final prediction model in this example, by combining classification and regression, we were able to take advantage of the property that prediction accuracy improves when NS substances are removed from the data. As shown above, the comparison of Models R1-R4 indicates that it is better to remove NS substances, and this can also be inferred from the fact that NS substances do not inherently have an EC3 value. In addition to this combined approach, by defining the application limit based on the accuracy of the ITSv2 hazard assessment, the prediction accuracy of Model F2 for the external validation data improved to an R2 value of "0.824" and an RMSE of "0.430".
[0074] In addition, in Models R6, R8, R6', and R8', which are components of Models F1, F1', and F2, the results were consistent with previous reports that the h-CLAT values are useful for prediction, and the importance of various variables, such as variables based on the read-across concept and molecular descriptors that represent the distribution of valence electrons in the chemical structure, was also shown. Skin sensitizers are generally considered to have electrophilicity, and these variables may express the reactivity of chemicals, including their electrophilicity (OECD, 2022a).
[0075] In addition, for multi-class classification using predicted EC3 values, the concordance rates for external validation data for ITSv2, Model F1, and F2 in three-class classification based on the UN GHS classification were 0.667, 0.744, and 0.879, respectively. In Model F2, a high concordance rate was also achieved in classification by setting the scope of application. However, caution is required because the data sets evaluated in Model F1 and Model F2 are different due to the exclusion of misclassified substances in the ITSv2 hazard assessment.
[0076] Quantitative prediction of EC3 values greatly improves the degree of freedom in evaluation, such as UN GHS classification and comparison of sensitization strength. For example, when considering appropriate exposure concentrations in human evaluations such as HRIPT (Human Repeated Insult Patch Test), the exposure concentration cannot be quantitatively considered only by prior evaluation based on classification. Prediction of PoD (Point of Departure) for risk assessment using in silico models is expected to play an important role in the future in the implementation of skin sensitization evaluation (Api et al., 2020; Natsch et al., 2014). [Industrial Applicability]
[0077] Using a machine learning model and ITSv2 hazard, we constructed an in silico evaluation system that quantitatively predicts LLNA EC3 values. By specifying the scope of application and evaluating the prediction performance of external data using external validation data, we were able to construct a more robust prediction model. We are confident that the model construction, evaluation, and limitation design in this invention will contribute to the progress of in silico toxicity prediction evaluation research and contribute to the skin sensitization evaluation of chemicals. [Explanation of symbols]
[0078] 1. Information Processing Systems 2 Processing equipment 3 Memory means 4 Information display section 5. Input methods 21 Distribution Processing Unit 22 Prediction model construction processing section 23 Output Processing Section 31 Objective variable storage section 32 Explanatory variable storage section
Claims
1. An information processing system for predicting a quantitative indicator of a positive event caused by a chemical substance, comprising: a response variable storage unit that stores information on a quantitative index of a positive event caused by a chemical substance as a response variable; an explanatory variable storage unit that stores at least one of experimental information, information on a chemical structure, and information on a substance similar to the test substance as an explanatory variable; a sorting unit that sorts and stores information on the objective variables and explanatory variables into learning data and verification data; a prediction model construction processing unit that constructs a prediction model of the index using the objective variable and explanatory variables of the learning data by a machine learning technique, evaluates the prediction model using the objective variable and explanatory variables of the validation data, and reconstructs the prediction model as necessary; An information processing system comprising an output processing unit that uses the prediction model to output the predicted value of the index of a chemical substance.
2. The information processing system according to claim 1 , wherein the predictive model construction processing unit evaluates the predictive accuracy of the predictive model and the presence or absence of overlearning, using the objective variable and explanatory variables for the validation.
3. The information processing system according to claim 2 , wherein the prediction model construction processing unit evaluates the prediction accuracy and determines whether or not overlearning has occurred based on an R value and an RMSE value calculated by the following formulas: [0010]
4. The allocation processing unit further allocates the verification data to two types of data: internal verification data and external verification data; The information processing system according to claim 3 , wherein the prediction model construction processing unit determines the presence or absence of overlearning using the internal validation data, and evaluates prediction accuracy using the external validation data.
5. 2. The information processing system according to claim 1, wherein the allocation processing unit compresses information on each of the objective variables and explanatory variables by normalizing and standardizing the information to match the scale (number of digits), and then allocates the information to the learning data and the verification data and stores the data.
6. The information processing system according to claim 1 , wherein the prediction model construction processing unit uses a result of a correct / incorrect ITSv2 hazard evaluation as a threshold for an application range.
7. 2. A control program for causing a computer to function as the information processing system according to claim 1, the control program causing a computer to function as the allocation processing unit, the prediction model construction processing unit, and the output processing unit.
8. An information processing method executed by a system that predicts a quantitative indicator of a positive event caused by a chemical substance, the system comprising: a response variable storage unit that stores, as a response variable, information of a quantitative indicator of a positive event caused by a chemical substance; and an explanatory variable storage unit that stores, as an explanatory variable, at least one piece of information selected from experimental information, information on a chemical structure, and information on a substance similar to a test substance, the system comprising: a classification step of classifying and storing information on the objective variables and explanatory variables into learning data and verification data; a prediction model construction step of constructing a prediction model for the value of the index using the objective variable and explanatory variables of the learning data by a machine learning technique, evaluating the prediction model using the objective variable and explanatory variables of the validation data, and reconstructing the prediction model as necessary; and an output step of outputting the predicted value of the index of a chemical substance using the prediction model.
