Method for evaluating a prediction of a result of a property of a molecule, and associated device

Abstract

The invention relates to a method for evaluating a prediction of a result of a property of a molecule referred to as the predicted molecule, the property being one among genotoxicity, mutagenicity and carcinogenicity, the method comprising the following steps: obtaining (S100) the prediction of the result of the property of the predicted molecule, the prediction being determined by a model on the basis of a first sub-set of a set of molecules; determining (S200) at least two values among a value representative of a consistency, for the property, of the results of the molecules of the sub-set with said prediction; a value representative of a co-similarity of the molecules within the sub-set; and a value representative of a capacity of the model to predict the result of the property of the predicted molecule; and determining (S300), by means of a fuzzy logic inference algorithm, a value representative of the reliability of the prediction of the property result of the predicted molecule.

Description

Description Title of the invention: Method for evaluating a prediction of a result of a property of a molecule, and associated device Technical Field

[0001] The present invention belongs to the general field of chemical molecules. It relates more particularly to a method for evaluating a result of at least one property of a molecule, the at least one property being one of genotoxicity, mutagenicity and carcinogenicity. It also relates to an electronic device configured to implement such a method.

[0002] The invention thus has a preferred but non-limiting application in the prediction of the genotoxicity, mutagenicity and / or carcinogenicity of organic or inorganic molecules constituting a large number of materials used industrially, such as for example composite materials, glues, varnishes, paints, energy materials for space propulsion. Prior art

[0003] The entry into force in 2007 of the European REACH regulation (acronym for "Registration Evaluation Authorisation of Chemicals") requires manufacturers in the European Economic Area who manufacture, import or use chemical substances in their activity in quantities greater than 1 tonne per year to register these substances at the European level. This involves identifying, evaluating and controlling all chemical substances manufactured, imported or placed on the European market. This regulation aims to provide the European Union with legal and technical means to guarantee a high level of protection against risks related to chemical substances. It concerns all chemical substances, whether inert materials (e.g. additives, stabilizers, plasticizers, glues, etc.) or energetic or even highly energetic.

[0004] There is therefore a need for manufacturers, in order to comply with this regulation in particular, to have techniques to identify the biological and / or toxicological effects that a chemical substance can produce on humans or the environment, and more generally to identify its properties in terms of genotoxicity, mutagenicity and / or carcinogenicity.

[0005] Genotoxicity is particularly important since it allows the assessment of the carcinogenic and mutagenic risk of the substance, possibly leading to its classification as "Carcinogenic, Mutagenic and Reprotoxic" (CMR). The development and selection of non-CMR substances is therefore a major concern for manufacturers who wish to avoid to see their substance seriously threatened by the restriction measures imposed by application of the REACH regulation.

[0006] In vitro or in vivo techniques exist, but they are generally long, complex to implement and very costly in terms of resources, reagents and detection methods.

[0007] There are also other so-called in silico techniques which rely on computer tools (e.g. computer models, computerized calculation methods) to predict the properties of a chemical substance.

[0008] These in silico techniques have the advantage of reducing the number of animals required for toxicological experimentation, but also the costs related to the implementation of testing strategies. The most common in silico techniques use "quantitative structure-activity relationships" (also called QSAR for Quantitative Structure Activity Relationship in English), which are algorithms (or equivalently programs) establishing a quantitative prediction of the effects of a variation in the molecular structure of a chemical substance on biological activity. These algorithms make it possible, for example, to determine a relationship between a molecular structure and properties in terms of mutagenicity.

[0009] The biological activity translated by QSARs is based on experimental results and is specific to a given test (for example the Ames test), typically correlated with the requirements defined by the REACH regulation and / or by the OECD / OCDE (Organization for Economic Cooperation and Development).

[0010] Alternatives to using QSAR exist, such as the supervised learning method ExtraTrees ("extremely randomized trees") which allows the construction of an ensemble of decision trees, or the RMoS algorithm.

[0011] However, the use of these in silico techniques raises either the problem of model reliability or the problem of correct interpretation of predictions. Moreover, even if a certain model has overall performance showing prediction accuracy above a certain threshold, the fact remains that this model may have significant local variations in performance.

[0012] There is therefore a need to evaluate these techniques in silico using measures describing the reliability of a prediction. Statement of the invention

[0013] The present invention aims to remedy all or part of the drawbacks of the prior art, in particular those set out above, by proposing a solution which makes it possible to evaluate the reliability of a prediction of a result of a property of a molecule.

[0014] To this end, and according to a first aspect, the invention relates to a method for evaluating a prediction of at least one property of a so-called predicted molecule, the at least one property being one of genotoxicity, mutagenicity and carcinogenicity. The method according to the invention comprises the following steps, implemented by an electronic device: - obtaining the prediction of the result of the at least one property of the predicted molecule, the prediction being determined by a model from a first subset of a set of molecules, each of the molecules of the first subset having, for the at least one property, a result determined experimentally, the molecules of the first subset having a measure of similarity with the predicted molecule greater than a predetermined threshold; - determination of at least two values ​​from a value representative of a consistency, for the at least one property, of the results of the molecules of a second subset with said prediction; a value representative of a cosimilarity of the molecules within the second subset; and a value representative of a capacity of the model to predict the result of the at least one property of the predicted molecule; - determination, by a fuzzy logic inference algorithm, of a value representative of the reliability of the prediction of the result of the at least one property of the predicted molecule, the fuzzy logic inference algorithm taking the determined values ​​as input.

[0015] More particularly, the present invention makes it possible to determine a domain of applicability for this predictive model, that is to say to determine, in the chemical space, the regions of this chemical space where the predictions of the model are reliable, and the regions where the predictions are not reliable. This domain of applicability is defined by the value representative of a consistency, the value representative of a cosimilarity and the value representative of a capacity of the model to predict a result. The limit of the domain of applicability is determined using this value representative of the reliability of the prediction, which is for example included in the interval [0; 10].

[0016] The invention thus provides a tool for implementing strategies and decision-making based on an analysis of the model's applicability domain, in order to increase the efficiency and relevance of the use of this predictive model. In this way, the present invention allows an industrialist to control the risk associated with the use of a given model.

[0017] No limitation is attached to the nature of the predicted molecule considered. For example, this is a highly energetic molecule (or HEM). However, this example is given for illustrative purposes only and the invention applies to all types of molecules.

[0018] Generally speaking, it is considered that the steps of a process should not be interpreted as being linked to a notion of temporal succession.

[0019] In particular embodiments, the first and second subsets are identical. Alternatively, the second subset is a subset of the first subset.

[0020] In particular modes of implementation, the evaluation method may further comprise one or more of the following characteristics, taken individually or in all technically possible combinations.

[0021] In particular modes of implementation, the representative value of the consistency, for the at least one property, of the results of the molecules of the second subset with said prediction is determined as a function of with n7 the number of molecules in the second subset having, for at least one property, a result identical to the prediction, and K the number of molecules in the second subset.

[0022] In particular modes of implementation, the representative value of the cosimilarity of the molecules within the second subset is determined as a function of ^=i- y K (M0 i) with S(MOLi) a value representative of the similarity of a molecule from the second subset with the predicted molecule and K the number of molecules from the second subset.

[0023] In particular modes of implementation, the representative value of the capacity is determined as a function of , with M a number of molecules of at least one part of the second subset), and with M' the number of molecules of the at least one part having, for the at least one property, a predicted result identical to the result determined experimentally.

[0024] In particular embodiments, the consistency of the results of the molecules, the cosimilarity of the molecules within the second subset, the capacity of the model to predict the result, and the reliability of the prediction of the result define input or output variables, each input or output variable being defined by a set of at least one membership function, the at least one membership function representing a degree of membership between a linguistic term characterizing a possible state of an input or output variable, and between a value range of said input or output variable, and the method further comprises obtaining, by the fuzzy logic inference system, said sets of at least one membership function.

[0025] In particular modes of implementation, the membership functions of the set defining the consistency of the results, of the set defining the capacity of the model, and of the set defining the reliability of the prediction are triangular-shaped functions, and the membership functions of the set defining the cosimilarity of the molecules within the second subset are trapezoidal functions.

[0026] In particular embodiments, the method further comprises obtaining a rule base, each rule defining a relationship between a possible state of at least one variable among the consistency of the results, the cosimilarity within the second subset and the capacity of the model, and between a possible state of the reliability of the prediction, the possible states being characterized by a linguistic term.

[0027] In particular embodiments, the method further comprises defuzzifying a degree of membership of the reliability of the prediction, so as to obtain a numerical value representative of said reliability, the defuzzification being implemented by applying a method among the center of the surface, the modified center of the surface, the center of the sums, the center of the maximum or the average of the maxima.

[0028] According to a second aspect, the invention relates to an electronic device configured to evaluate a prediction of an outcome of at least one property of a predicted molecule, the at least one property being one of genotoxicity, mutagenicity and carcinogenicity, the device comprising: - a module for obtaining the prediction of the result of the at least one property of the predicted molecule, the prediction being determined by a model from a first subset of a set of molecules, each of the molecules of the first subset having, for the at least one property, a result determined experimentally, the molecules of the first subset having a measure of similarity with the predicted molecule greater than a predetermined threshold; - a module for determining at least two values ​​from among a value representative of a consistency, for the at least one property, of the results of the molecules of a second subset with said prediction; a value representative of a cosimilarity of the molecules within the second subset; and a value representative of a capacity of the model to predict the result of the at least one property of the predicted molecule; - an evaluation module configured to determine, by a fuzzy logic inference algorithm, a value representative of the reliability of the prediction of the result of the at least one property of the predicted molecule, the fuzzy logic inference algorithm taking the determined values ​​as input.

[0029] According to a third aspect, the invention relates to a computer program comprising instructions for implementing a method for evaluating a prediction of a result according to the invention, when said program is executed by a processor.

[0030] According to a fourth aspect, the invention relates to a computer-readable recording medium on which the computer program according to the invention is recorded. Brief description of the drawings

[0031] Other characteristics and advantages of the present invention will emerge from the description given below, with reference to the appended drawings which illustrate an exemplary embodiment thereof without any limiting character. In the figures:

[0032] [Fig.l] Figure 1 schematically represents a particular mode of implementation of an electronic device for evaluating a prediction as proposed;

[0033] [Fig. ] Figure 2 schematically represents an example of hardware architecture of the electronic device of [Fig.l];

[0034] [Fig.3] Figure 3 represents, in the form of a flowchart, a particular mode of implementation of a method for evaluating a prediction of the result of a property of a molecule as proposed, implemented in a system comprising the electronic device of [Fig.l];

[0035] [Fig.4] Figure 4 represents an example of sets of membership functions for the input and output variables of the fuzzy logic inference algorithm;

[0036] [Fig.5] Figure 5 is a graphical representation of a prediction error rate as a function of a reliability value. Description of the embodiments

[0037] Figure 1 schematically represents a particular mode of implementation of an electronic device for evaluating a prediction as proposed.

[0038] Figure 2 schematically represents an example of hardware architecture of an electronic device 10 for evaluating a prediction.

[0039] As illustrated in Figure 2, the electronic device 10 for evaluating a prediction has the hardware architecture of a computer. Thus, the electronic device 10 comprises, in particular, a processor 1, a random access memory 2, a read-only memory 3 and a non-volatile memory 4. It also comprises a communication module 5.

[0040] The read-only memory 3 of the electronic device 10 constitutes a recording medium as proposed, readable by the processor 1 and on which is recorded a computer program PROG in accordance with the invention, comprising instructions for the execution of steps of the method for evaluating a prediction as proposed below. The program PROG defines functional modules of the device as represented in FIG. 1, which rely on or control the hardware elements 1 to 5 cited previously, and which include in particular: - a MODJDBT module for obtaining the PRED prediction of the result of at least one property of a so-called predicted molecule, the prediction being determined by a model from a first SSENS subset of a set of molecules, each of the molecules of the first subset having, for the at least one property, a result determined experimentally, the molecules of the first subset having a measure of similarity with the predicted molecule greater than a predetermined threshold; - a MOD_DET module for determining at least two values ​​from a value representative of a consistency, for the at least one property, of the results of the molecules of a second subset with said prediction; a value representative of a cosimilarity of the molecules within the second subset; and a value representative of a capacity of the model to predict the result of the at least one property of the predicted molecule; and, - an evaluation module MOD_EVAL configured to determine, by a fuzzy logic inference algorithm, a value representative of the reliability of the prediction of the result of the at least one property of the predicted molecule, the fuzzy logic inference algorithm taking as input the at least two determined values.

[0041] In particular implementation modes, the MOD_EVAL evaluation module includes in particular the following sub-modules: - a MOD_FUZZ fuzzification submodule configured for: - obtaining a plurality of sets of at least one membership function, each set of the plurality defining a criterion among the consistency of the results, the cosimilarity of the molecules within the second subset, and the capacity of the model to predict the result of the at least one property of the predicted molecule, - obtaining a value representative of the consistency of the results of the molecules of the second subset with said prediction; a value representative of a cosimilarity of the molecules within the second subset; and a value representative of a capacity of the model to predict the result of the at least one property of the predicted molecule; and, - associate the at least two values ​​obtained with a membership function of the plurality of sets. - a MOD_INF sub-module configured to apply a plurality of fuzzy rules. More precisely, this MOD_INF sub-module applies each fuzzy rule so as to obtain a partial conclusion corresponding to a degree of membership with a linguistic term characterizing a possible state of the reliability of the prediction of the predicted molecule. This partial conclusion is then aggregated with the other partial conclusions resulting from the application of the other rules of the plurality. - a MOD_DEF defuzzification sub-module configured to determine, from the aggregated partial conclusions, a value representative of the reliability of the prediction of the predicted molecule.

[0042] Furthermore, the electronic device 10 may also comprise other modules, in particular for implementing particular modes of the method for evaluating a prediction, as described in more detail later.

[0043] The communication module 5 notably allows the electronic device 10 for evaluating a prediction to access or download the content of one or more databases each listing a plurality of molecules.

[0044] The databases may be hosted on remote servers or at least part of their content may be stored in a memory of the electronic device 10 for evaluating a prediction (for example in its non-volatile memory 4). The communication module 5 of the electronic device 10 for evaluating a prediction allows it to access or download it via a telecommunications network, or to obtain these databases via a recording medium such as a USB (Universal Serial Bus) key or a CD-ROM. They may include for this purpose a USB port, a network card, a WIFI (Wireless Fidelity) interface, etc.

[0045] Figure 3 represents, in the form of a flowchart, a particular mode of implementation of a method for evaluating a prediction of the result of a property of a molecule as proposed, implemented in a system comprising the electronic device of [Fig.l].

[0046] As illustrated in Figure 3, the method for evaluating a PRED prediction of a property of a MOLP molecule comprises a first step S100 of obtaining the PRED prediction of the result of the property of the predicted MOLP molecule. This prediction is determined by a model from a first SSENS subset from a set ENS of molecules, each of the molecules of the first SSENS subset having, for this property, a result determined experimentally, and the molecules of the first SSENS subset having a similarity measure with the predicted MOLP molecule greater than a predetermined threshold THR1. This first step S100 is implemented by the module MODJDBT for obtaining the prediction of the electronic device 10.

[0047] The similarity between two molecules encoded in the form of vectors A and B is a well-known notion in itself, and which is for example quantifiable through different measures and criteria such as: - the Euclidean distance d Euc (A, B) = iSJZi G4[i] — B [i]), with n the number of elements of vectors A and B, - the distance from Manhattan - I distance index ^Jac (fî) — 1 ijac (, B), - the distance measurement defined in patent FR3068047 Bl: = y Oety > 0 Oety = 0 otherwise where x denotes the value of an element of a vector representing a first molecule and y the value of an element of a vector representing a second molecule (called current).

[0048] The first obtaining step S100 comprises the sub-steps S110, S120 and S130. During step S110, the molecule whose property is to be predicted is obtained. Then, during a step S120, a first reference SSENS subset is obtained from a database comprising a set ENS of molecules each associated with the experimental result achieved by this molecule in a given biological test.

[0049] Examples of such biological tests include the Ames test (a biological test for determining the mutagenic potential of a chemical substance), the HPRT test, the glycophorin A test, the comet assay (or "Cornet assay"), the sister chromatid exchange test, the chromosomal aberration assessment test, the micronucleus test, etc.

[0050] For the sake of simplification, we will subsequently consider the case where a single database is accessed which comprises a set ENS of molecules and the experimental results achieved by these molecules in a given biological test. However, the invention also applies in the case where several databases are considered.

[0051] Such databases are known per se and are not described in detail here. Each database corresponds, for example, to a biological test carried out on the molecules it contains. Examples of these databases are described in particular in the document by DJ. Kirkland et al., entitled "Testing strategies in mutagenicity and genetic toxicology: an appraisal of the guidelines of the European Scientific Committee for Cosmetics and Non-Food Products for the evaluation of hair dyes", Mutât. Res. Toxicol. Environ. Mutagen, vol. 588, pages 88-105, 2005, or in the document by V. Thybaud et al. entitled "Strategy for genotoxicity testing: hazard identification and risk assessment in relation to in vitro testing", Mutât. Res. Toxicol. Environ. Mutagen, vol. 627, pages 41-58, 2007.

[0052] As mentioned previously, the database may be hosted on remote servers or its contents may be stored in a memory of the electronic device 10 for evaluating a prediction (for example in its non-volatile memory 4). The communication module 5 of the electronic device 10 for evaluating a prediction allows it to access it, or to download its contents via a telecommunications network, or to obtain the contents of these databases via a recording medium such as a USB (Universal Serial Bus) key or a CD-ROM. They may include for this purpose a USB port, a network card, a WIFI (Wireless Fidelity) interface, etc.

[0053] It is important to note at this stage that the algorithms for predicting the outcome of a property discussed below are based on a search for structural similarity between the ENS set of molecules listed in the database and the molecule for which the outcome of a property is to be predicted. This similarity search is based on the assumption that all molecules in the ENS set that are analogous to the molecule in question have similar properties.

[0054] In particular modes of implementation, this step S120 of determining a first reference SSENS subset is implemented in the following manner: - determination, for each of the molecules in the ENS set of molecules in the database, of a representative measure of the similarity with the molecule whose properties we wish to predict; and, - selection, from the ENS set, of a first SSENS subset of molecules which present a minimal similarity with the molecule whose properties we wish to predict.

[0055] Alternatively, this step S120 of determining a first reference SSENS subset is implemented by identifying a subset of molecules from the database from another known set of molecules, for example a set of highly energetic molecules used by an industrialist, and selecting the molecules from the database which have minimal similarity with each of the molecules from the known set.

[0056] Alternatively, this step S120 of obtaining a first reference SSENS subset is a step of determining a reference SSENS subset implemented in accordance with the method which is the subject of patent FR3068047B1.

[0057] According to this particular mode of implementation, the obtaining step S120 is then implemented through an iterative method of determining a first SSENS subset of reference molecules. This iterative determination method comprises an initialization step associating, with a current molecule, a value of a predetermined molecule descriptor. associated with the molecule obtained during step SI 10, and during each iteration of the determination process: - a step of evaluating, for each molecule of a database comprising a plurality of molecules ENS each associated with a value of said descriptor, a measure of overall similarity between the value of the descriptor associated with said molecule and the value of the descriptor associated with the current molecule; - a step of selecting molecules from the database having an overall similarity measure greater than a predetermined threshold (eg, 0.85), the selected molecules being added to the reference SSENS subset; and - a step of updating the value of the descriptor associated with the current molecule from the values ​​of the descriptors associated with at least part of the molecules belonging to the first SSENS reference subset.

[0058] Thus, this step S120 of determining a first SSENS subset is based on an iterative similarity search process, initialized firstly with the molecule whose properties are to be predicted. Then, over the course of the iterations, “virtual” molecules are constructed from the descriptors of the molecules selected in the initial database during the iterations, and a new similarity search is carried out from these virtual molecules.

[0059] The method for evaluating a prediction of a property further comprises a step S130 during which the result of at least one property of the molecule obtained during step S110 is predicted from the first subset obtained during step S120. In other words, a prediction technique is used which considers the experimental results achieved by the molecules of the first reference SSENS subset and listed in the database from which the first SSENS subset was extracted.

[0060] No limitation is attached to the prediction technique implemented. It may in particular be a QSAR type algorithm previously mentioned, which makes it possible to establish a quantitative prediction of the effects of a variation in the molecular structure of a chemical substance on the biological activity. Alternatively, a learning technique (e.g., a neural network, a support vector machine, a method based on decision trees, a k-nearest neighbor method, etc.), or a prediction technique by principal component analysis (or PCA for "Principal Component Analysis") or by partial least squares (or "Partial Least Squares") may be considered.

[0061] At the end of step S130, a prediction of at least one biological property of the molecule obtained during step S110 is obtained.

[0062] In the embodiments described so far, steps S110, S120 and S130 are performed by the electronic device 10 for evaluating a prediction. Alternatively, these steps are implemented by an electronic prediction device separate from the electronic device 10 for evaluating a prediction, and step S100 of obtaining the prediction PRED of the at least one property of the predicted molecule MOLP is a step of receiving, from the electronic prediction device, this prediction. According to this variant, the electronic device 10 for evaluating a prediction is also configured to access the first reference SSENS subset determined by the prediction device during step S120 of obtaining a first SSENS subset.

[0063] The method for evaluating a prediction further comprises a step S200 during which several metrics representative of the first reference SSENS subset used to predict the property of the predicted molecule MOLP are calculated. More precisely, at least two of the following values ​​are determined during this step: a value representative of a consistency of the properties of the molecules of a second subset with said prediction (PRED); a value representative of a cosimilarity within the second subset; and a value representative of a capacity of the model to predict the at least one property of the predicted molecule (MOLP). This step S200 is implemented by the MOD_DET module for determining the electronic device 10.

[0064] Value representing the consistency of the properties of the molecules of a subset with said prediction

[0065] We recall at this stage that the molecules of the first SSENS subset have a certain degree of similarity with the predicted molecule (MOLP), and that the results of the properties of the molecules of this first SSENS subset were determined experimentally (and were therefore not predicted), and that they therefore constitute a ground truth.

[0066] In this example, the second subset consists of the K molecules from the first subset that are most similar to the molecule whose properties we wish to predict.

[0067] The consistency of the properties of the molecules of the second subset with said PRED prediction is determined by a statistical analysis of the results of the properties of the K molecules of the second subset. More specifically, it is determined whether the PRED prediction made during step S130 is consistent with the experimental results of these K molecules.

[0068] Intuitively, if for example it is predicted that a given molecule is not genotoxic, and a majority of the experimental results of the K molecules in the second subset actually reflect that they are not genotoxic, then this means that the said PRED prediction is consistent with the results of the properties of the molecules in the second subset. Conversely, if it is predicted that a given molecule is not genotoxic, but a majority of the experimental results of these K molecules reflect that they are genotoxic, then this means that the said PRED prediction is not consistent with the results of the properties of the molecules in the second subset.

[0069] Thus, in particular modes of implementation, the value I c representative of the consistency, for the at least one property, of the results of the molecules of the second subset with said PRED prediction is determined so that

[0070] with rij the number of molecules in the second subset having, for a given property (for example genotoxicity), an identical result (for example positive) to that predicted during step S120, and K the number of molecules in the second subset. The value l cis representative of the fact that if the result of the prediction is consistent with the experimental results of the K molecules most similar to the molecule whose properties we wish to predict, then there is a greater probability of seeing the prediction verified.

[0071] The inventors observed that the evolution of the value l c representative of consistency is correlated with that of the error rate œ c Or

[0072] More precisely, a value l c high is representative of a low error rate. Thus, for a value l c = 1, the error rate is equal to 2.4%. For a value l c included in the interval [0.9; 1[, the error rate œ c is 7.4%. However, for a value of c low (i.e. included in the interval [0; 0.1[), the error rate œ c corresponding is 35.5%. Also, these results show that the value lc is insensitive to detecting an unreliable prediction.

[0073] Value I representing a cosimilarity within a subset

[0074] The cosimilarity of molecules within the second subset aims to characterize the concentration of similar molecules among themselves in this second subset. This criterion is advantageous since it makes it possible to discriminate the case where the subset does not include enough convincing examples to predict the result of a property. However, without a sufficient number of convincing examples, the result of a given property of a molecule cannot be correctly predicted.

[0075] Thus, in particular modes of implementation, the value I d representative of the cosimilarity within the second subset is determined so that with S MOLi) a value representative of the similarity of a molecule (MOLj) of the second subset with the predicted molecule (MOLP) and K the number of molecules of the second subset.

[0076] A similarity measure S(MOLi) is defined as a real number between 0 and 1, taking by convention the value 0 when the two molecules are considered totally different (i.e. not similar), and the value 1 when they are considered totally identical (i.e. similar). Intermediate values ​​can be considered, representing nuances of similarity between these two extremes.

[0077] To comply with this definition, a so-called conversion function is applied to any of the previously mentioned distance measures, which converts the distance measure d into a value in the interval [0,1].

[0078] The applied conversion function f verifies for example (d) = exp (^2) , with d the distance to be converted, and o a predetermined real number.

[0079] The inventors observed that a value I d representative of a cosimilarity lower than 0.70 reflects an inability to correctly predict the target molecule having such an index value, since in this case, the error rate œ c is equal to 1. In other words, a low value means that there are no compounds in the molecular environment close to the target molecule that are strongly or moderately similar to it, which is problematic during the learning phase of the predictive model.

[0080] When the value I d representative of a cosimilarity is included in the interval [0.65; 0.70 [, the error rate is equal to 1. On the other hand, a value l drepresentative of a cosimilarity strictly greater than 0.70 reflects a capacity to provide a correct prediction, since the error rate is less than or equal to 0.15.

[0081] Ability of the model to predict the outcome of the predicted molecule property

[0082] The ability of the model to predict the outcome of the property of the predicted molecule corresponds, for example, to determining the ability of the model to predict the toxicity of a chemical substance with a certain molecular structure. This criterion aims on the one hand to discriminate the predictions made by the model when the latter presents poor performance in terms of predicting results for molecules of the molecular family to which the predicted molecule belongs, and on the other hand to make reliable those made by the model when the latter presents good performance in terms of predicting results for molecules of the molecular family to which the predicted molecule belongs.

[0083] This criterion aims to grant confidence in the predictions made by the model when it presents good performance in predicting the molecular family to which the predicted molecule belongs.

[0084] The determination of this capacity of the model to predict a result of a property of a molecule similar to the predicted molecule is implemented in the following manner: - during a first step, at least a portion of the molecules of the first SSENS subset is selected, so as to obtain a second subset. According to a particular implementation, the molecules selected during this first step correspond to the K molecules most similar to the predicted molecule, with K=20 for example. Alternatively, the molecules selected during this first step correspond to the molecules having a similarity measure with the predicted molecule greater than a predetermined threshold THR2, for example such that THR2 > THR1. Finally, in a particular case, all the molecules of the first SSENS subset are selected, and the first and second subsets are then identical; - then, for each of the molecules selected during this first step, a result is predicted, by the model, for the property considered; - finally, a value I P representative of the capacity is determined such that: with M the number of molecules in the second subset, and M' the number of molecules in the second subset which have, for at least one property, a predicted result identical to the result obtained experimentally.

[0085] The inventors observed that when the value I P representative of the capacity is high, the model is able to correctly predict the molecular family of the substance to be predicted and therefore we can have confidence in the capacity of said model to predict the latter.

[0086] More precisely, when the value I P is strictly less than 0.5, the error rate œ cvaries between 75 and 90% error. And when the value I P is equal to 1, the error rate œ c is about 4.3%.

[0087] Returning to [Fig. 3], the method according to the invention further comprises a step S300 of determining, by a fuzzy logic inference algorithm, a value VALF representative of the reliability of the prediction of the result of the at least one property of the predicted molecule MOLP. This step S300 is implemented by the module MOD_EVAL for evaluating the result of the prediction of the electronic device 10.

[0088] The fuzzy logic inference algorithm takes as input the values ​​determined during step S200. Its particularity is to be able to determine a numerical output value based on a set of rules formulated in natural language.

[0089] Using a fuzzy logic inference algorithm to evaluate the prediction of the outcome of a molecular property offers the advantage of reasoning not on numerical variables, but on linguistic variables, which allow the manipulation of knowledge in natural language.

[0090] Furthermore, when this system is used in a regulatory context by manufacturers wishing to justify the biological and / or toxicological effects that a chemical substance can produce on humans or the environment, the use of a fuzzy logic inference algorithm facilitates, for humans, the interpretation of the conclusions which can be carried out by analysis / reading of the rule base.

[0091] Concerning the generic principles of fuzzy logic, those skilled in the art can refer to the work "La logique floue, "Que sais-je?" n° 2702, Bernadette Bouchon-Meunier, 2007".

[0092] This step S300 of determining, by a fuzzy logic inference algorithm, a VALF value representative of the reliability of the prediction of the result comprises the sub-steps S310, S320 and S330.

[0093] The step referenced S310 corresponds to a step of fuzzification of the inputs of the fuzzy logic inference algorithm. This step S310 is implemented by the fuzzification sub-module MOD_FUZZ of the electronic device 10, and has the objective of converting a digital input data (for example a value representative of the capacity of the system to predict a result equal to 0.75) into one or more linguistic variables to which degrees of membership are associated (for example “high = 0.5” and “medium high = 0.5”). For this, the algorithm relies on so-called membership functions which are obtained by the electronic device 10 during this step S310.

[0094] A membership function represents a degree of membership between a linguistic term characterizing a possible state of an input or output variable of the algorithm, and between a range of values ​​of said input or output variable.

[0095] The standard membership functions classically used are type A (triangular shape), type fl (trapezoidal shape), singleton type (vertical line shape), Sigmoid type (sinusoidal shape) or Gaussian type (bell shape).

[0096] Figure 4 shows an example of sets of membership functions for the input and output variables of the fuzzy logic inference algorithm.

[0097] In this example, the input variable corresponding to the consistency of the results takes the following states: "minimal", "very low", "low", "medium", "medium high", "high", "very high", "extreme" and "maximum". The "minimal" and "maximum" states are represented by a membership function of type n, and the other states by a membership function of type A. Thus, for the "average" state, the associated membership function is equal to 0 on the interval [0; 0.4] U [0.6; 1], is increasing from 0 to 1 on the interval [0.4; 0.5], and is decreasing from 1 to 0 on the interval [0.5; 0.6].

[0098] The input variable corresponding to the model's ability to predict an outcome takes the following states: "very low", "low", "medium", "medium high", "high", "very high" and "maximum". The "very low" and "maximum" states are represented by a membership function of type fl, and the other states by a membership function of type A.

[0099] The input variable corresponding to the cosimilarity of molecules within the subset (SSENS) takes the following states: “low” and “high”. These states are represented by a membership function of type fl. Thus, the membership function associated with the “low” state is equal to 1 on the interval [0; 0.7] and to 0 on the interval [0.7; 1]. And the function associated with the “high” state is equal to 0 on the interval [0; 0.7] and to 1 on the interval [0.7; 1].

[0100] Finally, the output variable of the model corresponds to the prediction reliability of the result, and takes the following states: "minimal", "very low", "low", "medium low", "medium", "medium high", "high", "very high", " " and "maximum". The states "very very low" and "maximum" are represented by a membership function of type fl, and the other states by a membership function of type A.

[0101] Returning to Figure 3, step S310 of fuzzifying an input also comprises obtaining at least two values ​​from the following: a value representative of the consistency of the results of the molecules of the subset with said prediction; a value representative of a cosimilarity of the molecules within the subset; and a value representative of a capacity of the model to predict the result of the at least one property of the predicted molecule. For example, the value I p representative of the consistency of the results I p = 0.45, a value I d representative of a cosimilarity I d = 0.65, and the value I p representative of the system's ability to predict an outcome I p = 0.75.

[0102] These numerical values ​​are then "fuzzified", that is to say that each of them is redefined as a set of fuzzy data, each fuzzy data corresponding to a degree of membership in a possible state of the corresponding input variable.

[0103] Thus, considering the membership functions represented by Figure 4A, the representative value of the system's ability to predict a result I p = 0.75 is redefined as: "high capacity = 0.5"; "medium high capacity = 0.5"; and the other possible capacity states are instantiated to a zero value.

[0104] The evaluation method further comprises an inference step S320 corresponding to the application of a “rule base”. This step S320 is implemented by the inference sub-module MOD_INF of the electronic device 10.

[0105] A rule in a rule base is typically of the form "If condition Then conclusion". The condition (also called the premise of the rule) is defined by a relation of the form "input variable is possible state" or by a combination of relations i = 1. . n of the form "input variable [i] is possible state of ia input variable [i]" using fuzzy logic operators. The conclusion (also called the consequence of the rule) is defined by a relation of the form "output variable is possible state".

[0106] The fuzzy operators typically used in this framework are: - the operator "and" which is defined such that X and Y = min(XK) with mi ( ) the minimality operator; or X and Y = X. Y with . the product operator; - the “or” operator which is defined such that X or Y = max(XK) with max ) the maximality operator; or X or Y = 1 — (1 — a). (1 — b); and, - the “no” operator.

[0107] When implementing this invention, a rule base of 59 rules was defined. These rules formalize the behavior of the value (VALF) representative of the reliability of the prediction as a function of the representative value of a consistency, the representative value of a cosimilarity and the representative value of a capacity of the model to make a prediction.

[0108] As previously described, the evolution of the representative value of a coherence and the evolution of the representative value of a capacity of the model to make a prediction are correlated with the evolution of the error rate. On the other hand, the representative value of a cosimilarity has a profile of the "threshold value" type. This behavior is for example translated by the following rule: "If the representative value of a cosimilarity is low, then the representative value of the reliability of the prediction is minimal."

[0109] "If the representative value of a model's ability to make a prediction is low and the representative value of consistency is very low, then the representative value of the reliability of the prediction is very very low." is another example of a rule defined in this framework.

[0110] During this inference step S320, each fuzzy rule of the rule base is applied, so as to obtain a partial conclusion corresponding to a degree of membership with a linguistic term characterizing a possible state of the reliability of the prediction of the predicted molecule. [YES] Step S330 corresponds to a defuzzification step. This step is implemented by the MOD_DEF sub-module of the electronic device 10, and takes place in two stages.

[0112] First of all, when the rule base considered uses several inference rules which have the same conclusion (for example, several inference rules have as conclusion "reliability of prediction of the result is moderately low"), the application of these rules makes it possible to obtain conclusions which must be combined. It is classically the "or" operator defined as XorY = max X,' K) which is used to implement this step.

[0113] Thus, if three rules give the following partial conclusions: "reliability of prediction of the result is moderately low" at 20%; "Ability of prediction of the result is moderately low" at 45%; "reliability of prediction of the result is moderately low" at 10%, the combination of these rules using the "or" operator previously defined allows us to obtain the conclusion "reliability of prediction of the result is moderately low" at 45%.

[0114] This step allows obtaining a unique degree of membership for at least part of the possible states of the output variable.

[0115] Then, in a second step, the degrees of membership associated with the different possible states of the output variable are "defuzzified", that is to say they are converted into a single numerical value VALF representative of the reliability of the prediction of the result of the at least one property of the predicted molecule MOLP is determined.

[0116] Different methods can be used to implement this step, such as center of surface, modified center of surface, center of sums, center of maximum or average of maxima.

[0117] Figure 5 is a graphical representation of a prediction error rate as a function of a reliability value.

[0118] Through this figure 5, a very good correlation (0.9) is observed between the error rate and the reliability value, when considering all the points. We note that the higher the reliability value, the less probability there is that the prediction is false. A reliability value between 0 and 1 excluded corresponds to an error rate of 88%. While a reliability value between 9 and 10 excluded corresponds to an error rate of 2%.

[0119] The method according to the invention is therefore advantageous since, unlike other methods, such as class probabilities, the lowest value classes correspond to a high error rate and the highest value classes correspond to a very low error rate.

[0120] Up to now, the invention has been described in the case where only one prediction is implemented. However, the invention also remains applicable in the case where several predictions are made from one or more databases storing the results of several biological tests.

Claims

Claims

1. Method for evaluating a prediction (PRED) of a result of at least one property of a molecule (MOLP) said to be predicted, the at least one property being one of genotoxicity, mutagenicity and carcinogenicity, the method comprising the following steps, implemented by an electronic device (10): - obtaining (S100) the prediction (PRED) of the result of the at least one property of the predicted molecule, the prediction (PRED) being determined by a model from a first subset (SSENS) of a set (ENS) of molecules, each of the molecules of the first subset (SSENS) having, for the at least one property, a result determined experimentally, the molecules of the first subset (SSENS) having a measure of similarity with the predicted molecule greater than a predetermined threshold; - determination (S200) of at least two values ​​from a value representative of a consistency, for the at least one property, of the results of the molecules of a second subset with said prediction (PRED); a value representative of a cosimilarity of the molecules within the second subset; a value representative of a capacity of the model to predict the result of the at least one property of the predicted molecule (MOLP), the second subset corresponding to the first subset or to a portion of the first subset; - determination (S300), by a fuzzy logic inference algorithm, of a value (VALF) representative of the reliability of the prediction of the result of the at least one property of the predicted molecule (MOLP), the fuzzy logic inference algorithm taking the determined values ​​as input.

2. Evaluation method according to claim 1, in which the value representative of the consistency, for the at least one property, of the results of the molecules x , , , n j of the second subset with said prediction (PRED) is determined as a function of — , K with nj the number of molecules of the second subset having, for at least one property, a result identical to the prediction (PRED), and K the number of molecules of the second subset.

3. An evaluation method according to claim 1 or 2, wherein the representative value of the cosimilarity of the molecules within the second subset is determined based on avec s(M0Li) a representative value of the similarity of a molecule MOLt) of the second subset with the predicted molecule (MOLP) and K the number of molecules of the second subset.

4. An evaluation method according to any one of claims 1 to 3, in M' which the representative value of the capacity is determined as a function of — , with M a number of molecules of at least one part of said second subset, and with M' the number of molecules of the at least one part having, for the at least one property, a predicted result identical to the result determined experimentally.

5. An evaluation method according to any one of claims 1 to 4, wherein the consistency of the results of the molecules, the cosimilarity of the molecules within the second subset, the ability of the model to predict the result, and the reliability of the prediction (PRED) of the result define variables, each variable being defined by a set of at least one membership function, the at least one membership function representing a degree of membership between a linguistic term characterizing a possible state of a variable, and between a value range of said variable, the method further comprising obtaining (S310), by the fuzzy logic inference system, said sets of at least one membership function.

6. The evaluation method of claim 5, wherein the membership functions of the set defining the consistency of the results, of the set defining the capacity of the model, and of the set defining the reliability of the prediction (PRED) are triangular-shaped functions, and the membership functions of the set defining the cosimilarity of the molecules within the second subset are trapezoidal-shaped functions.

7. An evaluation method according to claim 5 or 6, further comprising obtaining a rule base, each rule defining a relationship between a possible state of at least one variable among the consistency of the results, the cosimilarity within the second subset and the capacity of the model, and between a possible state of the reliability of the prediction, the possible states being characterized by a linguistic term.

8. An evaluation method according to any one of claims 5 to 7, wherein the method further comprises defuzzifying (S330) a degree of membership of the reliability of the prediction (PRED), so as to obtain a numerical value representative of said reliability, the defuzzification being implemented by applying a method among the center of the surface, the modified center of the surface, the center of the sums, the center of the maximum or the average of the maxima.

9. Electronic device (10) configured to evaluate a prediction (PRED) of a result of at least one property of a predicted molecule (MOLP), the at least one property being one of genotoxicity, mutagenicity and carcinogenicity, the device comprising: - a module (MODJDBT) for obtaining the prediction of the result of the at least one property of the predicted molecule, the prediction (PRED) being determined by a model from a first subset (SSENS) of a set (ENS) of molecules, each of the molecules of the first subset (SSENS) having, for the at least one property, a result determined experimentally, the molecules of the first subset (SSENS) having a measure of similarity with the predicted molecule (MOLP) greater than a predetermined threshold; - a module (MOD_DET) for determining at least two values ​​from a value representative of a consistency, for the at least one property, of the results of the molecules of a second subset with said prediction (PRED); a value representative of a cosimilarity of the molecules within the second subset; and a value representative of a capacity of the model to predict the result of the at least one property of the predicted molecule (MOLP), the second subset corresponding to the first subset or to a portion of the first subset; - an evaluation module (MOD_EVAL) configured to determine, by a fuzzy logic inference algorithm, a value representative of the reliability of the prediction of the result of the at least one property of the predicted molecule (MOLP), the fuzzy logic inference algorithm taking the determined values ​​as input.

10. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to implement the evaluation method according to any one of claims 1 to 8.

Images